Error in imputation_by_samples_posterior_expectation

[jhu99]

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells).
The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot.
I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts：
Error in imputation_by_samples_posterior_expectation(data, selected_logxx, :
Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix))
system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdf
zerorateincell.pdf

Could you give me a guide to setup the parameter?

best,
jialu

[ChenMengjie]

Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>.

[jhu99]

Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ChenMengjie]

Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>.

[jhu99]

Hi Mengjie， Thanks. I guess every gene has at least one zero. I will check it today and let you know. It’s 10x data for 293t cell line from Zheng’s publication. Best, Jialu 发自我的 iPhone

…

在 2019年3月1日，上午12:03，ChenMengjie ***@***.***> 写道： Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer ***@***.******@***.***>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer ***@***.******@***.******@***.***>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jhu99]

Hi Mengjie, Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ChenMengjie]

Hi Jialu, Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways. Best, Mengjie On Mar 1, 2019, at 8:58 AM, screamer <[email protected]<mailto:[email protected]>> wrote: Hi Mengjie, Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd>.

[jhu99]

Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways. Best, Mengjie On Mar 1, 2019, at 8:58 AM, screamer <[email protected]<mailto:[email protected]>> wrote: Hi Mengjie, Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ChenMengjie]

No problem. Please let me know if you have comments or suggestions on the method. We hope to improve VIPER so that it can be used in more settings. -Mengjie On Mar 14, 2019, at 3:48 PM, screamer <[email protected]<mailto:[email protected]>> wrote: Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways. Best, Mengjie On Mar 1, 2019, at 8:58 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Hi Mengjie, Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417VBTDXuTY4ikVsrBY1rq3ARyIk59ks5vWrWDgaJpZM4bSUxd>.

[jhu99]

I am happy to make some contribution to improve VIPER if possible. Recently, my work focuses on data alignment for single cell RNA-seq (classification for 10x data. I am open to discuss on any of these two problems. Best Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Thursday, March 14, 2019 4:52 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) No problem. Please let me know if you have comments or suggestions on the method. We hope to improve VIPER so that it can be used in more settings. -Mengjie On Mar 14, 2019, at 3:48 PM, screamer <[email protected]<mailto:[email protected]>> wrote: Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways. Best, Mengjie On Mar 1, 2019, at 8:58 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: Hi Mengjie, Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset? Thanks, Mengjie On Feb 27, 2019, at 2:22 PM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Thanks Mengjie! You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply. Best, Jialu Sent from Mail for Windows 10 From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1) Hi Jialu, It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells? Thanks, Mengjie

…

Sent from my iPhone On Feb 26, 2019, at 7:38 AM, screamer <[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>> wrote: Hello Mengjie, I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step. The code is written as below: system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt)) zerorateingene.pdf<https://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf> zerorateincell.pdf<https://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf> Could you give me a guide to setup the parameter? best, jialu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP417VBTDXuTY4ikVsrBY1rq3ARyIk59ks5vWrWDgaJpZM4bSUxd>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.