Have to take three CLEP exams within 10 days. I'm overwhelmed!

[KittenMittens]

I believe the military get their first test for free, which gives them less of an incentive to study hard. The percentage of people who fail after using IC is unknown, so those numbers are almost meaningless. We don't know which way the sampling bias swings and how much it swings.

If they were meaningless, then so would Gallup polls, or any other survey which intrinsically have some level of sampling bias and error. If 95 out of 100 people in a given city, county, and state preferred one politician over the other, then is that meaningless? In the case of instantcert test takers, we already know what our sample set is and it doesn't reflect the entire test taker population for any given exam.

Statistics isn't perfect, but it's damn useful at least when used properly and it's up to the reader to determine what is or isn't within the acceptable range of statistical uncertainty. Example - 2008 election where Nate Silver successfully predicted how each state would vote based on "limited" sample spaces.

It's safe to say that if out of 10 people, 6 passed, that there's some greater level of uncertainty. But out of 720 people, 95% passed? That's meaningful. It's almost equally likely (50/50 chance) that someone either will or will not report their score, but let's say there's a slight bias towards those who pass to report their score so let's say 60/40 or 70/30 will report their data. Even then, it doesn't account for the fact that such a large percentage passed. If successful and unsuccessful test takers were roughly equally likely to report their scores, then we should see a much lower pass rate than 95%. Not to mention that Clep Difficulty List - Free-Clep-Prep.com indicates that it's of moderate difficulty (for the typical CLEP exam which again is much easier than the other exams available), which is a source that investigates what the aggregate forum opinion is, and of course student feedback which can be tallied to see what trend exists. Most importantly, we can at least compare different InstantCert exams to see what the trends are.

All statistical models are inherently flawed, but some more-so than others. That's why there's always going to be some level of uncertainty about going into something, but it is definitely not meaningless.

[sanantone]

If they were meaningless, then so would Gallup polls, or any other survey which intrinsically have some level of sampling bias and error. If 95 out of 100 people in a given city, county, and state preferred one politician over the other, then is that meaningless? In the case of instantcert test takers, we already know what our sample set is and it doesn't reflect the entire test taker population for any given exam.

Statistics isn't perfect, but it's damn useful at least when used properly and it's up to the reader to determine what is or isn't within the acceptable range of statistical uncertainty. Example - 2008 election where Nate Silver successfully predicted how each state would vote based on "limited" sample spaces.

It's safe to say that if out of 10 people, 6 passed, that there's some greater level of uncertainty. But out of 720 people, 95% passed? That's meaningful. It's almost equally likely (50/50 chance) that someone either will or will not report their score, but let's say there's a slight bias towards those who pass to report their score so let's say 60/40 or 70/30 will report their data. Even then, it doesn't account for the fact that such a large percentage passed. If successful and unsuccessful test takers were roughly equally likely to report their scores, then we should see a much lower pass rate than 95%. Not to mention that Clep Difficulty List - Free-Clep-Prep.com indicates that it's of moderate difficulty (for the typical CLEP exam which again is much easier than the other exams available), which is a source that investigates what the aggregate forum opinion is, and of course student feedback which can be tallied to see what trend exists. Most importantly, we can at least compare different InstantCert exams to see what the trends are.

All statistical models are inherently flawed, but some more-so than others. That's why there's always going to be some level of uncertainty about going into something, but it is definitely not meaningless.

The Gallup Poll does not depend on self-selection. That's the difference. When you do some form of randomized, stratified, or systematic sampling, there is less bias than depending on self-selection. We do not know if only 720 IC users took the test because we do not know how many people have chosen not to report their score. You can't just guess how many people choose to report vs. not report without doing some digging. Randomized sampling means that you are likely to get a sample that is mostly representative of the population, depending on self-selection does not.

[KittenMittens]

The Gallup Poll does not depend on self-selection. That's the difference. When you do some form of randomized, stratified, or systematic sampling, there is less bias than depending on self-selection. We do not know if only 720 IC users took the test because we do not know how many people have chosen not to report their score. You can't just guess how many people choose to report vs. not report without doing some digging. Randomized sampling means that you are likely to get a sample that is mostly representative of the population, depending on self-selection does not.

Self-selection merely makes it more difficult to ascertain the causation of something, but it can at least help identify a correlation which is still very useful. We can compare different InstantCert subjects, for example, and track them to see trends. For instance, some of the IC subjects are tracking and passing at clearly lower percentages and that correlates with the overall total population rates for those subjects.

Sure, self-selected samples are not scientifically cogent or precise, but they provide something to base inferences from and worth examining. It's not an exact science, and there's a certain level of "gut instinct" and intuition you have to rely on. It's not perfect, but it's better than nothing and gives a good idea.

When looking at all the FY 14 CLEP Civilian Pass rates, essentially every exam with the lowest % pass rates tend to track with the instantcert ones that also are lower than typically seen. For example, CLEP College Algebra is at 59% population rate, and tracks with instantcert's college algebra rates which is at 82% - both of which are lower than usual. DSST Money and Banking is another one which has a 55% official pass rate, and is also much lower on instantcert as well .

We can infer that DSST & CLEP exams for instance that have lower pass rates are going to have lower pass rates on InstantCert as well and that is indeed the case. So yes there is bias involved, but the inferences derived from both sets of data, and student feedback are certainly better than nothing at all. We can certainly extrapolate with some level of awareness, that InstantCert has a tangible effect on markedly improving scores.

I would hypothesize it's simply because people that are using IC materials are more likely to become prepared. I'd also predict that enough people are reporting that they're either passing or failing to at least fathom some questions and identify a correlation with population pass rates. We cannot ascertain precisely what percentage of IC test-takers are passing and failing unless everyone was required to report, but I'd wager that most people would find the information useful for their own purposes. At least the data correlates with the official rates, and the pass rates tend to be higher, which one could infer is due to IC preparing them well. One could easily test that and see if the results are statistically significant i.e. one group that uses IC vs another that uses nothing vs another that uses a textbook and see how each one fares. It would be kind of tedious and an exercise in futility however since most people already assume/believe that test prep is superior in general.

[sanantone]

Self-selection merely makes it more difficult to ascertain the causation of something, but it can at least help identify a correlation which is still very useful. We can compare different InstantCert subjects, for example, and track them to see trends. For instance, some of the IC subjects are tracking and passing at clearly lower percentages and that correlates with the overall total population rates for those subjects.

Sure, self-selected samples are not scientifically cogent or precise, but they provide something to base inferences from and worth examining. It's not an exact science, and there's a certain level of "gut instinct" and intuition you have to rely on. It's not perfect, but it's better than nothing and gives a good idea.

When looking at all the FY 14 CLEP Civilian Pass rates, essentially every exam with the lowest % pass rates tend to track with the instantcert ones that also are lower than typically seen. For example, CLEP College Algebra is at 59% population rate, and tracks with instantcert's college algebra rates which is at 82% - both of which are lower than usual. DSST Money and Banking is another one which has a 55% official pass rate, and is also much lower on instantcert as well .

We can infer that DSST & CLEP exams for instance that have lower pass rates are going to have lower pass rates on InstantCert as well and that is indeed the case. So yes there is bias involved, but the inferences derived from both sets of data, and student feedback are certainly better than nothing at all. We can certainly extrapolate with some level of awareness, that InstantCert has a tangible effect on markedly improving scores.

I would hypothesize it's simply because people that are using IC materials are more likely to become prepared. I'd also predict that enough people are reporting that they're either passing or failing to at least fathom some questions and identify a correlation with population pass rates. We cannot ascertain precisely what percentage of IC test-takers are passing and failing unless everyone was required to report, but I'd wager that most people would find the information useful for their own purposes. At least the data correlates with the official rates, and the pass rates tend to be higher, which one could infer is due to IC preparing them well. One could easily test that and see if the results are statistically significant i.e. one group that uses IC vs another that uses nothing vs another that uses a textbook and see how each one fares. It would be kind of tedious and an exercise in futility however since most people already assume/believe that test prep is superior in general.

No, it can't even be used for correlation. It could be that 2,000 people took a particular CLEP and of the half who didn't report, most of them failed. That would drastically change the pass rate. Yes, there is some correlation between the IC scores and official CLEP scores only in the assumed relative difficulty of individual tests in comparison to others, but a 90% pass rate makes an exam look extremely easy whereas a 60% pass rate would indicate some significant studying might be needed. The point I'm trying to get across is that you might be able to tell one exam is easier than another, but that still doesn't tell you how difficult that one exam is.

As far as being more prepared, that may or may not be true. We don't know how many people quit IC to turn to another study source. That, again, introduces self-selection bias because the only people using the flashcards are the people who like to study with flashcards. But, all of this really doesn't help the OP, and we're taking the thread off topic.