Nutrient profiling (NP) is defined as the science of ranking foods according to their nutritional composition for the purpose of preventing disease or promoting health. The application of NP is ultimately to assist consumers to make healthier food choices, and thus provide a cost effective public health strategy to reduce the incidence of diet-related chronic disease. To our knowledge, no review has assessed the evidence to confirm the validity of NP models. We conducted a systematic review to investigate the construct and criterion-related validity of NP models in ranking food according to their nutritional composition for the purpose of preventing disease and promoting health. We searched peer-reviewed research published to 30 June 2015 and used PUBMED, Global Health (CABI), and SCOPUS databases. Within study bias was assessed using an adapted version of the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies -2) tool for all diagnostic studies and the Cochrane Collaboration's Risk of Bias tool for all non-diagnostic studies. The GRADE (Grades of Recommendation, Assessment, Development, and Evaluation) approach was used to guide our judgement of the quality of the body of evidence for each outcome measure. From a total of 83 studies, 69 confirmed the construct validity of NP models; however most of these studies contained methodological weaknesses. Six studies used objective external measures to confirm the criterion-related validity of NP models; which inherently improved quality. The overall quality of evidence on the accuracy of NP models was judged to be very low to moderate using the GRADE approach. Many carefully designed studies to establish both construct and criterion-related validity are necessary to authenticate the application of NP models and provide the evidence to support the current definition of NP.