# Differential expression

### Simulated data

1. Implement a subroutine generating random data stemming either from the same or from different distributions, either by copying code from this snippet or by implementing similar functionality yourself. Check in your code into your github repository.
2. Generate a testset of 10000 analytes under two different conditions measured in 3 replicates. As you will note the data generator will generate two data frames, one containing the simulated expression values for all samples-replicate tuples (e.g. s2_1) in columns, and all analytes in rows (e.g. a12). There is also a row for sample number in this data frame. In addition, there is a data frame containing which poulation each analyte and sample combination was stemming from. s1 is alway from population 0, while s2 is either from population 0 or 1.
3. For each analyte, calculate a p-value, i.e. the probability that a difference between the two conditions was observed despite they stem from the same distribution, using a t-test the function `scipy.stats.ttest_ind`
4. Implement a python subroutine for estimating q-values from a set of p-values. Follow the steps of Remark B in Storey&Tibshirani. However instead of a spline estimation in step 2, just calculate the average of the pi_0 estimates with lambda >= 0.75.
5. Plot the (simulated) number of differential expression genes as a function of q-value threshold.
6. Plot the difference between q-values and the actual fraction of null statistics (using the labels from the generator).
7. (*) Generate a test set using 3 different conditions under triplicates
8. (*) Implement a 1-way ANOVA test to calculate p-values for no differences between the three conditions. Use statsmodels
9. (*) Repeat step 5 and 6 for this more complex testset.
10. Check in your code to your github repository.

I posted my version of parts of the exercise here.

### Weightloss blood plasma set

1. Download the expression data from the article “Proteomics reveals the effects of sustained weight loss on the human plasma proteome”
2. Import the excel file into a pandas data frame using e.g. the command
```import pandas as pd df = pd.read_excel("MSB-12-901-s009.xlsx",skiprows=list(range(6))+list(range(15,18)),parse_cols=+list(range(3,318)),header=0,index_col=0)```
3. Test which analytes that are significantly different from Timepoint 1 to Timepoint 7 for all the patients which were present for the full experiment series. Try to reuse as much as possible from the code from your experiments with simulated data, point 4-6. Do an implementation using (a) a regular t-test e.g. `scipy.stats.ttest_ind` (b) using a paired t-test using e.g. `scipy.stats.ttest_rel`. Make sure you understand the difference between the two tests, and why they give different results.
4. (*) Implement an ANOVA to test significant difference for any of the 7 points in time. In analogy with the paired t-test in the previous step, introduce a categorical blocking variable for the patients, i.e use an “Expression ~ C(Time) + C(Patient)” model. (Note that you will need to rename the tested gene product name to an easier parsable name at the time of testing, e.g. “Expression”)
5. (**) Implement an ANOVA to test the influence on weight loss on plasma expression. That is “Expression ~ C(Patient) +dW”, where dW is calculated as the difference in weight of the patient since time 1, i.e. dW=0 at time 1.
6. Check in your code to your github repository.

### CHO-cell growth data

Here we have an yet unpublished dataset on protein concentrations in the supernatant of a CHO-cell bioreactor for protein concentrations. The concentrations have been determined by so-called iBAQ spectral counting.

2. Read in the data to a pandas data frame. Tip: use a command like `df = pd.read_excel("CHOPER_IBAQ_sep.xlsx",header=0,index_col=0)`
4. Test which proteins that are differentially expressed between day 10and 30 at FDR 1% using a regular t-test e.g. `scipy.stats.ttest_ind`