Back to PLS HelpPosted on 11/28/22 02:27:43
Number of posts: 14
To whomever this might concern,
I have run a behavior PLS using 36 behavioral measures and got 2 significant latent variables (LV). For both LVs, some of the behavior correlations (orig_corr) lie outside of the upper and lower bounds as calculated by the bootstrapping (ulcorr and llcorr). Please see below the sample results for a few of such behavioral measure for one latent variable.
| orig_corr | ulcorr | llcorr | ulcorr_adj | llcor_adj |
| -0.10906 | -0.13331 | -0.21737 | -0.10215 | -0.10215 |
| -0.13448 | -0.16314 | -0.24496 | -0.13435 | -0.13435 |
| -0.13606 | -0.15863 | -0.24106 | -0.12028 | -0.12028 |
I have searched the forum and seemed like this has happened before. Any idea of how to best tackle this problem would be much appreciated!
Replies:
Posted on 12/01/22 15:24:40
Number of posts: 394
To whomever this might concern,
I have run a behavior PLS using 36 behavioral measures and got 2 significant latent variables (LV). For both LVs, some of the behavior correlations (orig_corr) lie outside of the upper and lower bounds as calculated by the bootstrapping (ulcorr and llcorr). Please see below the sample results for a few of such behavioral measure for one latent variable.
| orig_corr | ulcorr | llcorr | ulcorr_adj | llcor_adj |
| -0.10906 | -0.13331 | -0.21737 | -0.10215 | -0.10215 |
| -0.13448 | -0.16314 | -0.24496 | -0.13435 | -0.13435 |
| -0.13606 | -0.15863 | -0.24106 | -0.12028 | -0.12028 |
I have searched the forum and seemed like this has happened before. Any idea of how to best tackle this problem would be much appreciated!
Hi Eric,
This is a known issue for which we haven't found a good solution. It does happen with relatively small sample sizes but that does not mean the point estimates (correlations) are unreliable.
An easy fix is to use the bootstrap standard error to compute the confidence intervals rather than the percentile (whicih is what we do). That can be calculate from the field boot_result.distrib, which has the entire distribution for the behavior side of the analysis.
hope that helps
Randy
I'm Online
Posted on 12/01/22 15:59:18
Number of posts: 229
You might also want to check these outputs: - the fact that the *adj limits are identical is odd
num_LowVariability_behav_boots: display numbers of low
variability resampled hehavior data in
bootstrap test
badbeh: display bad behav data that is caused by
bad re-order (with 0 standard deviation)
which will further cause divided by 0
countnewtotal: count the new sample that is re-ordered
for badbeh
Posted on 12/02/22 00:30:01
Number of posts: 14
Dear all,
Thank you for your replies!
The data I used have large sample size (N=3000+) so I don't think that would be an issue here. The behavior data is however in Likert scale (0-5) and skewed toward 0, which might be a problem? However, I did the PLS analysis with similar behavior data with much smaller sample size (N=100+) without encountering such issues.
Regarding the fact that the *adj limits are identical, I did a quick check for the 1st LV. Below are all the troublesome corr which lies out of bound, quite a few of them do have identical adj limits, but some also have diffrent adj limits.:
| orig_corr | ulcorr | llcorr | ulcorr_adj | llcor_adj |
| -0.10906 | -0.13331 | -0.21737 | -0.1021512 | -0.1021512 |
| -0.13448 | -0.16314 | -0.24496 | -0.1343458 | -0.1343458 |
| -0.13606 | -0.15863 | -0.24106 | -0.1202817 | -0.1202817 |
| -0.11955 | -0.14697 | -0.22594 | -0.1142138 | -0.1142138 |
| -0.07491 | -0.08849 | -0.16343 | -0.0486365 | -0.0651988 |
| -0.09647 | -0.11874 | -0.19807 | -0.0954431 | -0.0954431 |
| -0.11839 | -0.1381 | -0.21888 | -0.1041022 | -0.1059555 |
| -0.09412 | -0.11646 | -0.20263 | -0.0924281 | -0.0924281 |
| -0.08441 | -0.10899 | -0.20175 | -0.0738357 | -0.0738357 |
| -0.07165 | -0.08109 | -0.16333 | -0.0490865 | -0.0666741 |
| -0.07854 | -0.09344 | -0.16793 | -0.0602779 | -0.0685815 |
| -0.0935 | -0.1022 | -0.18938 | -0.0592278 | -0.0903954 |
| -0.09013 | -0.11226 | -0.20569 | -0.0806942 | -0.0807793 |
| -0.10346 | -0.12547 | -0.20638 | -0.0822778 | -0.0822778 |
| -0.08193 | -0.09083 | -0.16295 | -0.0654829 | -0.0811599 |
| -0.08694 | -0.09354 | -0.17455 | -0.0548671 | -0.0838504 |
| -0.08051 | -0.08216 | -0.1496 | -0.0568116 | -0.0872811 |
| -0.08605 | -0.10171 | -0.17889 | -0.0703527 | -0.0703527 |
| -0.07481 | -0.08818 | -0.17049 | -0.0449204 | -0.0725828 |
I have checked num_LowVariability_behav_boots, badbeh and countnewtotal. All these show either 0 or [], which is normal?
I'm looking forward to seeing if you have any other ideas why this issue might have happened and how to address them!
Much thanks!
Posted on 12/06/22 20:48:27
Number of posts: 394
Dear all,
Thank you for your replies!
The data I used have large sample size (N=3000+) so I don't think that would be an issue here. The behavior data is however in Likert scale (0-5) and skewed toward 0, which might be a problem? However, I did the PLS analysis with similar behavior data with much smaller sample size (N=100+) without encountering such issues.
Regarding the fact that the *adj limits are identical, I did a quick check for the 1st LV. Below are all the troublesome corr which lies out of bound, quite a few of them do have identical adj limits, but some also have diffrent adj limits.:
| orig_corr | ulcorr | llcorr | ulcorr_adj | llcor_adj |
| -0.10906 | -0.13331 | -0.21737 | -0.1021512 | -0.1021512 |
| -0.13448 | -0.16314 | -0.24496 | -0.1343458 | -0.1343458 |
| -0.13606 | -0.15863 | -0.24106 | -0.1202817 | -0.1202817 |
| -0.11955 | -0.14697 | -0.22594 | -0.1142138 | -0.1142138 |
| -0.07491 | -0.08849 | -0.16343 | -0.0486365 | -0.0651988 |
| -0.09647 | -0.11874 | -0.19807 | -0.0954431 | -0.0954431 |
| -0.11839 | -0.1381 | -0.21888 | -0.1041022 | -0.1059555 |
| -0.09412 | -0.11646 | -0.20263 | -0.0924281 | -0.0924281 |
| -0.08441 | -0.10899 | -0.20175 | -0.0738357 | -0.0738357 |
| -0.07165 | -0.08109 | -0.16333 | -0.0490865 | -0.0666741 |
| -0.07854 | -0.09344 | -0.16793 | -0.0602779 | -0.0685815 |
| -0.0935 | -0.1022 | -0.18938 | -0.0592278 | -0.0903954 |
| -0.09013 | -0.11226 | -0.20569 | -0.0806942 | -0.0807793 |
| -0.10346 | -0.12547 | -0.20638 | -0.0822778 | -0.0822778 |
| -0.08193 | -0.09083 | -0.16295 | -0.0654829 | -0.0811599 |
| -0.08694 | -0.09354 | -0.17455 | -0.0548671 | -0.0838504 |
| -0.08051 | -0.08216 | -0.1496 | -0.0568116 | -0.0872811 |
| -0.08605 | -0.10171 | -0.17889 | -0.0703527 | -0.0703527 |
| -0.07481 | -0.08818 | -0.17049 | -0.0449204 | -0.0725828 |
I have checked num_LowVariability_behav_boots, badbeh and countnewtotal. All these show either 0 or [], which is normal?
I'm looking forward to seeing if you have any other ideas why this issue might have happened and how to address them!
Much thanks!
I did a bit more digging and it does seem that many have found this problem with percentile method of computing CI's from bootstrap estimation. I am going to try a few other options - like Efron's BCA verision. Would you be interested in helping out? I can either send you test code or you can send the data with the weird CI estimations
Posted on 12/07/22 21:05:00
Number of posts: 14
Hi Randy,
Thank you so much for your reply! I'd love to help out. If you can send me the test code so that I can try it out it would be great!
Posted on 12/09/22 14:50:11
Number of posts: 394
Hi Randy,
Thank you so much for your reply! I'd love to help out. If you can send me the test code so that I can try it out it would be great!
Cool. Would you mind sending me the results file from your first example? I want to look at the histograms to see if the ideas I have will work. you can email me the link randy_mcintosh@sfu.ca
thanks
Randy
Posted on 12/09/22 21:52:49
Number of posts: 14
Dear Randy,
I have sent you the PLS mat file. Do let me know if you have any trouble accessing it.
Sophia
Posted on 12/09/22 21:53:33
Number of posts: 14
Dear Randy,
I have sent you the PLS mat file. Do let me know if you have any trouble accessing it.
Sophia
Posted on 12/11/22 14:28:31
Number of posts: 394
Thanks for the file. Getting the code together will take a while so in the interim, you can use the bootstrap SE to get the confidence intervals.
e.g.,
selv11=std(result1_test.boot_result.distrib(1,1,:))
ulCI=result1_test.boot_result.orig_corr(1,1)+(selv11*1.96)
llCI=result1_test.boot_result.orig_corr(1,1)-(selv11*1.96)
or depending on your plotting program you can enter the SE directly into the program
Login to reply to this topic.
