A question for you math/stats whizzes...

[DanT]

Originally Posted by Fat Elvis

DanT-

I really appreciate your help. It was late last night (late for me), so I really wasn't expressing myself very clearly. I'm not in a class now, and it has been over 25 years since I've taken a stats/probability class, so I've forgotten most everything. I think that is why the wording is pretty wonky. I was just looking at some old D&D dice that I had stumbled across while working in the basement, and for some reason it had me thinking about stats and probability. I think I worded things the way that I had because, in my mind, I was trying to generalize the concepts beyond the die/dice.

Perhaps you can clear some things up for me since my addled brain is a bit foggy. Lets assume that an event has a .25 probability of occuring; would the population size affect the mean and standard deviation? Perhaps it only affects variance? The smaller the population size, the greater the variance? This would be due to the Central Limit Theorem, correct?

I think Third Eye was getting what I was asking; I just did a poor job of asking the question. I have more questions though; I just need to relearn how to crawl before I start walking again with this....

In answer to your questions, you have to bear in mind that there is an important distinction between the population mean and the population standard deviation and the sample mean and the same standard deviation. The parameters that describe the population distribution are fixed constants that may or may not be known to the investigator. You asked me to suppose that an event has a 0.25 probability of occurring. OK, well you are now telling me the relevant POPULATION distribution--a Bernoulli with a probability of success of 0.25. Once I know that, I can compute the population mean, which will be 0.25 and the population standard deviation, which will be the square root of the variance, which equals the square root of 0.25 * ( 1 - 0.25 ).

Now, in real life, we often don't know what the parameters are for the relevant population distribution. We simply have access to a sample of observations from that distribution. We can use the sample to produce estimates of the unknown population parameters. Two such estimates are the sample mean and the sample standard deviation. These estimators have sampling distributions associated with them and those distributions do indeed depend on the sample size. Sample means based on a sample size of, say, 10, will vary more from sample to sample than would sample means based on a sample size of, say, 1000. The Central Limit Theorem pertains to the sampling distribution of the sample mean. It says that if the sample is from a population distribution that has a fixed finite mean and a finite population standard deviation, then the sampling distribution for the sample means can be approximated by a normal distribution, as the sample size gets larger and larger. So, for example, the Bernoulli distrbution has finite population means and standard deviations, so the Central Limit Theorem would apply to the sampling distribution of sample means for samples taken from that distribution. The sampling distribution for sample means based on a sample size of 10 will look sorta like a bell curve, if the population mean for the Bernoulli distribution is somewhere between, say, 0.30 and 0.70, but if you use sample sizes of 1,000 or more, then the sampling distribution for the means will really look very much like a bell curve, except for population means close to the edges, very low probability or very high probability events.

[DanT]

Originally Posted by DanT

In answer to your questions, you have to bear in mind that there is an important distinction between the population mean and the population standard deviation and the sample mean and the same standard deviation. The parameters that describe the population distribution are fixed constants that may or may not be known to the investigator. You asked me to suppose that an event has a 0.25 probability of occurring. OK, well you are now telling me the relevant POPULATION distribution--a Bernoulli with a probability of success of 0.25. Once I know that, I can compute the population mean, which will be 0.25 and the population standard deviation, which will be the square root of the variance, which equals the square root of 0.25 * ( 1 - 0.25 ).

Now, in real life, we often don't know what the parameters are for the relevant population distribution. We simply have access to a sample of observations from that distribution. We can use the sample to produce estimates of the unknown population parameters. Two such estimates are the sample mean and the sample standard deviation. These estimators have sampling distributions associated with them and those distributions do indeed depend on the sample size. Sample means based on a sample size of, say, 10, will vary more from sample to sample than would sample means based on a sample size of, say, 1000. The Central Limit Theorem pertains to the sampling distribution of the sample mean. It says that if the sample is from a population distribution that has a fixed finite mean and a finite population standard deviation, then the sampling distribution for the sample means can be approximated by a normal distribution, as the sample size gets larger and larger. So, for example, the Bernoulli distrbution has finite population means and standard deviations, so the Central Limit Theorem would apply to the sampling distribution of sample means for samples taken from that distribution. The sampling distribution for sample means based on a sample size of 10 will look sorta like a bell curve, if the population mean for the Bernoulli distribution is somewhere between, say, 0.30 and 0.70, but if you use sample sizes of 1,000 or more, then the sampling distribution for the means will really look very much like a bell curve, except for population means close to the edges, very low probability or very high probability events.

I should have said that when we compute statistics on observations from a Bernoulli, we code Successes as a 1 and Failure as a 0. So the sample mean is really just the proportion of successes among all of the observations. (The population mean, on the other hand, is the probability that an observation--i.e. the next observation from the distribution, for example--will be a success.)

[Fat Elvis]

Originally Posted by DanT

I should have said that when we compute statistics on observations from a Bernoulli, we code Successes as a 1 and Failure as a 0. So the sample mean is really just the proportion of successes among all of the observations. (The population mean, on the other hand, is the probability that an observation--i.e. the next observation from the distribution, for example--will be a success.)

I find this all very fascinating. I really appreciate your help and time explaining this to me.


When using a Bernoulli, would you use the same methods to calculate the observation of two (or three or more) successes (not necessarily consecutive)? Or is using a Bernoulli limited to coding a single success or failure?

[DanT]

Originally Posted by Fat Elvis

I find this all very fascinating. I really appreciate your help and time explaining this to me.


When using a Bernoulli, would you use the same methods to calculate the observation of two (or three or more) successes (not necessarily consecutive)? Or is using a Bernoulli limited to coding a single success or failure?

The Wikipedia entries on mathematical topics tend to be quite reliable. The page on the Bernoulli distribution mentions the appropriate related distribution that applies to your question, which is the Binomial. The Bernoulli is for a single trial. The Binomial pertains to the number of successes observed in n independent trials from the same Bernoulli distribution.

http://en.wikipedia.org/wiki/Bernoulli_distribution