Instance-Based Transfer Learning with Similarity-Aware Subject Selection for Cross-Subject SSVEP-Based BCIs

Ziwen Wang¹, Yue Zhang², Zhiqiang Zhang³, Sheng Quan Xie³, Alexander Lanzon¹, William P. Heath¹, Zhenhong Li^1,†

¹University of Manchester, ²University of Bath, ³University of Leeds, ⁴Bangor University IEEE Journal of Biomedical and Health Informatics ^†Corresponding Author: zhenhong.li@manchester.ac.uk

Abstract

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can achieve high recognition accuracy with sufficient training data. Transfer learning presents a promising solution to alleviate data requirements for the target subject by leveraging data from source subjects; however, effectively addressing individual variability among both target and source subjects remains a challenge. This paper proposes a novel transfer learning framework, termed instance-based task-related component analysis (iTRCA), which leverages knowledge from source subjects while considering their individual contributions. iTRCA extracts two types of features: (1) the subject-general feature, capturing shared information between source and target subjects in a common latent space, and (2) the subject-specific feature, preserving the unique characteristics of the target subject. To mitigate negative transfer, we further design an enhanced framework, subject selection-based iTRCA (SS-iTRCA), which integrates a similarity-based subject selection strategy to identify appropriate source subjects for transfer based on their task-related components (TRCs). Comparative evaluations on the Benchmark, BETA, and a self-collected dataset demonstrate the effectiveness of the proposed iTRCA and SS-iTRCA frameworks. This study provides a potential solution for developing high-performance SSVEP-based BCIs with reduced target subject data.

Methodology

Overview of the proposed iTRCA framework.

The figure illustrates the proposed iTRCA (instance-based task-related component analysis) framework. For the $i$-th stimulus, the shared information across subjects is captured in a common latent space (the blue block) where source instances and the target subject's individual template are maximally correlated through weight vector $\hat{\mathbf{w}}_i^{GS}$ and the spatial filter $\hat{\mathbf{w}}_i^{GT}$. Meanwhile, target subject's individual information is captured by the spatial filter $\hat{\mathbf{w}}_i^{T_a}$ based on TRCA. In the test stage, the feature $\rho_i$ is composed of the subject-general feature $\rho_{1,i}$ and the subject-specific feature $\rho_{2,i}$, and then combined across all sub-bands to formulate the feature $r_i$ used for SSVEP recognition.

iTRCA computes the task-related components (TRCs) of source subjects as source instances and then projects them into a latent space, where these source instances and the target subject's template exhibit maximal correlation. This latent space captures the underlying characteristics of SSVEP responses shared between source and target subjects, while accounting for the varying contributions of individual source subjects. Meanwhile, iTRCA captures target subject's individual characteristics using TRCA.

However, not all source subjects contribute positively to the target subject's performance. When the knowledge learned from source subjects have a detrimental effect on the target subject (e.g., reducing recognition accuracy), the transfer is regarded as negative transfer. To address this issue, the proposed iTRCA framework is further enhanced by the subject selection-based iTRCA (SS-iTRCA), which integrates a similarity-based subject selection strategy to identify appropriate source subjects for transfer based on their TRCs. This approach mitigates negative transfer by ensuring that only relevant source subjects contribute to the transfer learning process.

Main Results

Benchmark Dataset Results — **(a) Benchmark**

Self-collected Dataset Results — **(c) Self-collected**

Figure 1. The average recognition accuracy and ITR for TRCA, iTRCA and SS-iTRCA across all subjects for different $d$ on three datasets: (a) Benchmark, (b) BETA, and (c) Self-collected. Error bars represent SEM. Asterisks indicate the statistically significant difference between two algorithms ($*: p<0.05$, $**: p<0.01$, $***: p<0.001$, $****: p<0.0001$). Accuracy and ITR are analyzed by two-way and one-way repeated-measures ANOVA, respectively.

The figure shows the average accuracy and ITR across all subjects for $d$ ranging from 0.2 s to 1 s in 0.2 s increments, with $N_c = 9$, $N_{tb} = 3$, and $c_{lb} = 0.9$. Across all datasets, recognition accuracy of all algorithms increases monotonically with $d$, suggesting that longer data segments provide more reliable information for recognition when $d<1$ s.

On public datasets (Benchmark and BETA), iTRCA outperforms TRCA from 0.4 s onward, with SS-iTRCA further improving accuracy. However, the extent of improvement varies: on BETA, SS-iTRCA shows a more pronounced accuracy enhancement over iTRCA as $d$ increases, whereas on Benchmark, the accuracy difference is less significant. On the self-collected dataset, iTRCA consistently outperforms TRCA across all $d$ values, while SS-iTRCA achieves comparable accuracy to iTRCA, with no statistically significant differences observed. Regarding ITR, the results across all datasets show a consistent pattern: ITR increases rapidly from 0.2 s to 0.6 s, peaks around 0.8 s, and either stabilizes or slightly declines at 1 s. Unlike the monotonic increase in accuracy, ITR exhibits a growth-and-decline trend as $d$ increases, reflecting the trade-off between recognition speed and accuracy: longer $d$ enhances accuracy but also reduces speed, potentially lowering overall transfer efficiency. On public datasets, SS-iTRCA achieves the highest ITR from 0.4 s onward, TRCA the lowest, and iTRCA lies in between. On the self-collected dataset, SS-iTRCA and iTRCA show comparable ITR performance, while TRCA consistently lags behind across all $d$ values.

**(a)** Average recognition accuracy across all subjects

**(b)** Number of selected source subjects

**(c)** Selected source subjects for each target subject at different thresholds

Figure 2. Subject selection performance on Benchmark. (a) Average recognition accuracy across all subjects. (b) The number of selected source subjects where the scatter point represents the target subject and the gray bar represents the average number across 35 target subjects. (c) The number of selected source subjects for each target subject at different thresholds $c_{lb}$. For example, for target subject 1, the number of selected source subjects is 15, 12, 10, 7, and 3 at $c_{lb}=$ 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. For target subject 11, all source subjects are included, corresponding to the case where the subject selection strategy is not triggered.

Figure 2(a) reveals the occurrence of negative transfer. As $c_{lb}$ decreases ($c_{lb} \leq 0.8$), more source subjects with lower similarity to the target subject are included, leading to a decline in accuracy. This finding coincides with the negative transfer phenomenon, where dissimilar source data can hurt the target task (target subject's SSVEP recognition in this study), highlighting the importance of source subject selection. Moreover, it demonstrates that the designed TRC-based similarity metric can reflect the similarity between source and target subjects, thereby improving recognition accuracy even with fewer, but more relevant, source subjects.

Recognition accuracy comparison — **(a)** Recognition accuracy

Computation cost comparison — **(b)** Computation cost

Figure 3. Performance comparison between the proposed methods (SS-iTRCA and iTRCA) with existing transfer learning methods (transRCA, DGTF, and CSSFT). (a) Recognition accuracy, with a paired t-test conducted between iTRCA and transRCA. (b) Training and inference time for each method, where the test time corresponds to a block comprising 40 test trials in the benchmark dataset.

All experiments are conducted using MATLAB R2022a on a Windows 11 system, equipped with a 13th Intel Core i7-13700H CPU running at 2.40 GHz and an NVIDIA GeForce RTX 4070 Laptop GPU. The proposed frameworks consistently outperforms TransRCA, DGTF, and CSSFT across all $d$ values, with statistically significant differences. For both training and inference stages, SS-iTRCA, iTRCA and TransRCA exhibit significantly lower computational costs compared to CSSFT and DGTF. Although TransRCA achieves shorter training time than SS-iTRCA and iTRCA, there is no statistically significant difference in inference time among them, which is crucial for real-time signal recognition. Compared with accuracy-based subject selection method CSSFT, the proposed similarity-based subject selection strategy SS-iTRCA substantially reduces the computation cost.

BibTeX

@ARTICLE{11028588, author={Wang, Ziwen and Zhang, Yue and Zhang, Zhiqiang and Xie, Sheng Quan and Lanzon, Alexander and Heath, William P. and Li, Zhenhong}, journal={IEEE Journal of Biomedical and Health Informatics}, title={Instance-Based Transfer Learning with Similarity-Aware Subject Selection for Cross-Subject SSVEP-Based BCIs}, year={2025}, volume={}, number={}, pages={1-11}, doi={10.1109/JBHI.2025.3577813}}