precRecall.RdPrecision-recall curves are a standard way to analyze the
quality of a classifier, as are roc curves. The difference
between ROC and Precision-recall (PR) is that PR is mostly concerned with
what is happening at the top of the ranking, while ROC looks at the whole
ranking. The precision-recall method in ErmineJ is similar in intent to the
method popularized in “GSEA“, but uses standard precision-recall (GSEA
originally used Kolmogorov-Smirnov statistics, but was changed to use a
modified K-S statistic that makes it more precision-recall-like).
Like the ROC method, the PRC method uses only the ranks of the gene scores. That is, all it cares about is the ordering of items obtained by your gene scores (e.g., t-test or fold-change), but doesn’t use the information about the relative values of the scores.
PR-scoring is rather like the ORA method in that it is concerned with what is going on at the “top” of your list, but unlike ORA it does not use a threhold. Thus PR-scoring would be appropriate whenever you consider the ORA method appropriate. The PR method requires a resampling step, so it is slower to run than ORA.
Method overview taken from: http://erminej.msl.ubc.ca/help/tutorials/tutorial-precision-recall-scoring/
precRecall(
scores,
scoreColumn = 1,
bigIsBetter = FALSE,
logTrans = FALSE,
annotation = NULL,
aspects = c("Molecular Function", "Cellular Component", "Biological Process"),
iterations = 10000,
geneReplicates = c("mean", "best"),
pAdjust = c("FDR", "Bonferroni"),
geneSetDescription = "Latest_GO",
customGeneSets = NULL,
minClassSize = 20,
maxClassSize = 200,
output = NULL,
return = TRUE
)A data.frame. Rownames have to be gene identifiers (eg. probes,
must be unique), followed by any number of columns. The column used for
scoring is chosen by scoreColumn. See
http://erminej.msl.ubc.ca/help/input-files/gene-scores/ for
information abot how to specify scores. (for test = ORA, GSR and ROC)
Integer or character. Which column of the scores data.frame
to use as scores. Defaults to first column of scores. See
http://erminej.msl.ubc.ca/help/input-files/gene-scores/ for details.
(for test = ORA, GSR and ROC)
Logical. If TRUE large scores are considered to be higher.
FALSE by default (as in p values).
Logical. Should the data be -log10 transformed. Recommended for
p values. FALSE by default
Annotation. A file path, a data.frame or a platform short
name (eg. GPL127). If given a platform short name it will be downloaded
from annotation repository of Pavlidis Lab (https://gemma.msl.ubc.ca/annots/).
To get a list of available annotations, use listGemmaAnnotations.
Note that if there is a file or folder with the same name as the platform
name in the directory, that file will be read instead of getting a copy from
Pavlidis Lab. If this file isn't a valid annotation file, the function will fail.
If providing a custom annotation file, see makeAnnotation to do it from
R or erminej.msl.ubc.ca/help/input-files/gene-annotations/ to do it manually.
If you are providing a custom gene set, you can leave annotation as NULL
Character vector. Which Go aspects to include in the analysis.
Can be in long form (eg. 'Molecular Function') or short form (eg. c('M','C','B'))
Number of iterations. We suggest a starting value of 10000 iterations. When you decide on parameters you like, we recommend a larger number of iterations (perhaps 200,000 or more). This is to get sufficient precision in the p-values to make multiple-test correction work correctly. (test = GSR CORR and precRecall methods only)
What to do when genes have multiple scores in input file (due to multiple probes per gene)
Which multiple test correction method to use. Can be "FDR" or 'Westfall-Young' (slower).
"Latest_GO", a file path that leads to a GO XML or OBO file or a URL that leads to a go ontology file that ends with rdf-xml.gz.
If you left annotation as NULL and provided customGeneSets, this argument is
not required and will default to NULL. Otherwise, by default it'll be set to
"Latest_GO" which downloads the latest available GO XML file. This option won't work
without an internet connection. To get a frozen file
that you can use later, see goToday, goAtDate and getGoDates.
See http://erminej.msl.ubc.ca/help/input-files/gene-set-descriptions/
for details.
Path to a directory that contains custom gene set files, paths to custom gene set files themselves or a named list of character strings. Use this option to create your own gene sets. If you provide directory you can specify probes or gene symbols to include in your gene sets. See http://erminej.msl.ubc.ca/help/input-files/gene-sets/ for information about format for this file. If you are providing a list, only gene symbols are accepted.
minimum class size
maximum class size
Output file name.
If results should be returned. Set to FALSE if you only want a file
A list containing a "results" component and a "details" component. "results" is a data.frame containing the main output. The columns of this table are
Name: the name of the gene set
ID: the id of the gene set
NumProbes: the number of elements (e.g. probes) in the gene set.
NumGenes: the number of genes in the gene set.
RawScore: the raw statistic for the gene set. For explanations see this page
Pval: the p value for the gene set.
CorrectedPvalue: the corrected p pvalue. See this page for more information.
MFPvalue: pvalue after multifunctionality correction. Might be missing if correction was not performed.
CorrectedMFPvalue: Like CorrectedPvalue, but for the multifunctionality “corrected” pvalue.
Multifunctionality: How biased the genes in the set are towards multifunctional genes.
Same as: a list of gene sets which have the exact same members as this one. Such gene sets are not listed anywhere else.
GeneMembers: If you selected the “Include genes” option when saving, this will contain a list of the genes that are in the gene set, separated by “|”.
"details" section contain settings that were used to run the analysis.