CSS 10% ISSUES

FreeShoot

Hello,
I recently did some testing involving CSS10% and I discovered that the chance of the scrolls is not very faithful to your chance of success. See test images (Reliable test for 99.85%)



Please, tell me if you have a similar problem.

Neospector

You're using chi-squared distribution to calculate the overall probability?

χ² is the cumulative test statistic:
Σ((O-E)²/(E)
This can be used to calculate the probability of seeing x number of successes in n trials, but you're stopping after your first success, which is odd for this distribution. For a chi-squared distribution you should be using a fixed number of trials and observing the success rates in that number of trials.
If you're stopping after 1 success, You may wish to consider a Geometric distribution instead:

µ=1/p
σ² = (1−p)/p²
Where the probability of finding the first success after n trials is Σp(1−p)^(n−1)
Or, for this specific case:
0.1 * (0.9)^9 + 0.1 * (0.9)^8 + 0.1 * (0.9)^7 + 0.1 * (0.9)^6 + 0.1 * (0.9)^5 + 0.1 * (0.9)^4 + 0.1 * (0.9)^3 + 0.1 * (0.9)^2 + 0.1 * (0.9) + 0.1
Or a 0.6513215599 (~65% chance) that a scroll will succeed within the first 10 trials.

I'm not sure what your p-value is supposed to represent, usually a chi-squared test is useful in testing the probability that one particular mean is equal to, less than, or greater than another mean in another sample. There's not enough data presented to determine what's likely.
Which might be why you received a warning message stating your approximation may be incorrect.

FreeShoot

Neospector wrote: »

You're using chi-squared distribution to calculate the overall probability?

χ² is the cumulative test statistic:
Σ((O-E)²/(E)
This can be used to calculate the probability of seeing x number of successes in n trials, but you're stopping after your first success, which is odd for this distribution. For a chi-squared distribution you should be using a fixed number of trials and observing the success rates in that number of trials.
If you're stopping after 1 success, You may wish to consider a Geometric distribution instead:

µ=1/p
σ² = (1−p)/p²
Where the probability of finding the first success after n trials is Σp(1−p)^(n−1)
Or, for this specific case:
0.1 * (0.9)^9 + 0.1 * (0.9)^8 + 0.1 * (0.9)^7 + 0.1 * (0.9)^6 + 0.1 * (0.9)^5 + 0.1 * (0.9)^4 + 0.1 * (0.9)^3 + 0.1 * (0.9)^2 + 0.1 * (0.9) + 0.1
Or a 0.6513215599 (~65% chance) that a scroll will succeed within the first 10 trials.

I'm not sure what your p-value is supposed to represent, usually a chi-squared test is useful in testing the probability that one particular mean is equal to, less than, or greater than another mean in another sample. There's not enough data presented to determine what's likely.
Which might be why you received a warning message stating your approximation may be incorrect.

I'm not testing the probabiity, i'm testing the adherence of variables, exists a lot of qui-squared tests and p-vaue means the confiability of test(0.0015 means ¨the test is good for 99.85% of cases)