Bayesian Predictive Checking And Parameter Estimation

Finding an extreme p-value and thus ‘rejecting’ a model is never the end of an analysis;the departures of the test quantity in question from its posterior predictive distribution will often suggest improvements of the model or places to check the data, as in the speed of light example. Moreover, even when the current model seems appropriate for drawing inferences (in that no unusual deviations between the model and the data are found),

the next scientifific step will often be a more rigorous experiment incorporating additional factors, thereby providing better data. For instance, in the educational testing example of Section 5.5, the data do not allow rejection of the model that all the θj ’s are equal, but that assumption is clearly unrealistic, hence we do not restrict τ to be zero.Finally, the discrepancies found in predictive checks should be considered in their applied context.

Bayesian predictive checking generalizes classical hypothesis testing by averaging over the posterior distribution of the unknown parameter vector θ rather than fifixing it at some estimate θˆ. Bayesian tests do not rely on the construction of pivotal quantities (that is, functions of data and parameters whose distributions are independent of the parameters of the model) or on asymptotic results, and are therefore applicable in general settings. This is not to suggest that the tests are automatic; as with classical testing, the choice of test quantity and appropriate predictive distribution requires careful consideration of the type of inferences required for the problem being considered.

In the special case that the parameters θ are known (or estimated to a very high preci sion) or in which the test statistic T(y) is ancillary (that is, if it depends only on observed data and if its distribution is independent of the parameters of the model) with a continuous distribution, the posterior predictive p-value Pr(T(y rep > T(y) |y) has a distribution that isuniform if the model is true. Under these conditions, p-values less than 0.1 occur 10% of the time, p-values less than 0.05 occur 5% of the time, and so forth.

Bayesian Predictive Checking

• Given that the problem has geometric prior distribution with mean 100. The prior distribution is:

P(N)=(1/100)(99/100)^N-1

Given that the far number 203 is being observed here. Hence, it can be expected to be at least 203 cars. Any of the cars has equal probability to be the number of the last car. The likelihood function can be evaluated here as:

P(y/N) = (1/N), for each N>202,

• , else where.

The posterior distribution here will be:

P(N/y) α P( N)*P(y/N)

=(1/N)(0.01)(0.99)^N-1

α (1/N)(0.99)^N

• A probability distribution adds up to 1 and the posterior probability has to sum up to 1. Therefore, the normalizing constants here are:

P(X) = ∑(1/N)(0.99)^N      N=203(1) ∞

Therefore, the posterior mean is:

E[p(N/X)] = ∑_N=203¹⁰⁰⁰⁰ N = ∑_N=203^{10000 = 279.0885}

And, the standard deviation is:

sd[p(N/X)] = √ = 79.96458

N = 203:10000

p.x. = sum((1/N)*(99/100)*(N-1))

e.n. = sum(((99/100)*(N-1))/p.x)

e.n.

s.d.N = sqrt(sum((N-e.n.)^2*((1/N)*(99/100)*(N-1)/p.x.))

s.d.

• Let the problem has constant prior distribution. The prior distribution is:

P(N)=k.

Given that the far number 203 is being observed here. Hence, it can be expected to be at least 203 cars. Any of the cars has equal probability to be the number of the last car. The likelihood function can be evaluated here as:

P(y/N) = (1/N), for each N>202,

• , else where.

The posterior distribution here will be:

P(N/y) α P( N)*P(y/N)

=(1/N)*k

A probability distribution adds up to 1 and the posterior probability has to sum up to 1. Therefore, the normalizing constants here are:

P(X) = ∑(1/N)*k     N=203(1) ∞

Therefore, the posterior mean is:

E[p(N/X)] = ∑_N=203¹⁰⁰⁰⁰ N 

And, the standard deviation is:

sd[p(N/X)] = √

N = 203:10000

p.x. = k*sum(1/N)

e.n. = k*sum(1/p.x)

e.n.

s.d.N = k*sqrt(sum((N-e.n.)^2*((1/N)/p.x.))

s.d.Ns

Given is the that on survey on the number of bicycles and other vehicles recorded in an neighbourhood area of university of California, Berkeley on the division of two routes, one is for bike route and the other is non-bike route in an residential area. Given is the observed dataset:

A probability model is to be applied for this dataset for i=1(1)2. It can be easily said that the observed number of bicycles in the said area is binomial, that is,

y1,y2,...yi ~ ind Bin(ni,θi),

where, θ is the parameter of the model that resembles the proportion of  bike traffic at the jth location. The prior distribution for the parameters can be said to be beta distribution, that is ,

θi ~ iid Beta(α,β),

(b)

Therefore, the prior distribution here is :

P(θi) = Beta(α,β) α θiⁿⁱ(1-θ_i)^ni-θi

(c)

The likelihood function is:

P(yi /θi) α θi^(α-1)( 1-θ_i)^(β-1)

P(θi/yi) α P(θi)*P(yi/θi) α (θiⁿⁱ(1-θ_i)^ni-θi)*( θi^(α-1)( 1-θ_i)^(β-1)) = θi^(ni+α-1)(1- θi)^ni-θi+β-1

The required simulation result is:

α=0.152, β=0.220.

From popn 1:

P(yi /θ1) α θ1^(α-1)( 1-θ₁)^(β-1)

P(θi/yi) α P(θi)*P(yi/θi) α (θiⁿⁱ(1-θ_i)^ni-θi)*( θi^(α-1)( 1-θ_i)^(β-1)) = θi^(ni+α-1)(1- θi)^ni-θi+β-1 

=β(ni+α, ni-θi+β)

R codes are:

x=seq(0,1,length=100)

l=dbeta(x,10.152,9.9827)

l

plot(l)

For popn 2:

P(θ2) α θ2^(α-1)( 1-θ₂)^(β-1) ~ β(α, β)

P(θ2/yi) α β(n2+α, n2-θ2+β)

Simulated graph:

R codes are:

x2=seq(0,1,length=100)

k=dbeta(x1,8.152,8.1226)

k

 (d)

It can be said from the above plot that with a simulation of values with mean of first dataset minus mean of second dataset, their would not have been much difference in the plot of it. Therefore, it would not create much difference.   

The given data set is :

Residencial (with bike route): 16/58, 9/90, 10/48, 13/57, 19/103, 20/57, 18/86, 17/112, 35/273,55/64.

Residencial (without bike route): 12/113, 1/18, 2/14, 4/44, 9/208, 7/67, 9/29, 8/154.

The dataset are the  counts of bikes and other vehicles used, observed within a period of an hour in 10 blocks of a city. The dataset are being observed  in some residential area.

• Given is the that on survey on the number of bicycles and other vehicles recorded in an neighbourhood area of university of California, Berkeley on the division of two routes, one is for bike route and the other is non-bike route in an residential area. Given is the observed dataset:

A probability model is to be applied for this dataset for i=1(1)2. It can be easily said that the observed number of bicycles in the said area is binomial, that is,

y1,y2,...yi ~ ind Bin(ni,θi),

where, θ is the parameter of the model that resembles the proportion of  bike traffic at the jth location. The prior distribution for the parameters can be said to be beta distribution, that is ,

θi ~ iid Beta(α,β),  with hyper pior density as: p(α,β) =(α+β)^-2

Therefore, the prior distribution here is :

P(θi) = Beta(α,β) = ∏ θiⁿⁱ(1-θ_i)^ni-θi

And,the hyper prior distribution is:

P(α,β)=(α+β)^-(5/2)

The likelihood function is:

P(yi /θi) = ∏Cⁿⁱ_yi θi^(α-1)( 1-θ_i)^(β-1)

The posterior density is:

P(θi,α,β/yi) α P(θi)*P(yi/θi)*P(α,β) ⁼   ∏Cⁿⁱ_yi θi^(α-1)( 1-θ_i)^(β-1) *  ∏ θiⁿⁱ(1-θ_i)^ni-θi * (α+β)^-(5/2)

Therefore,

P(θi,α,β/yi) ~ (α+β)^{-(5/2) *}  * ∏θi^(α+yi-1)( 1-θ_i)^(β+ni-yi-1), that can also be written as:

P(θi,α,β/yi) ~ β(ni+α, ni-θi+β),

• The marginal distribution of α and β are:

P(α ,β /y) = 

The marginal density of the parameter is:

P(θ/α,β,y) = ∏ θi^(α+yi-1)( 1-θ_i)^(β+ni-yi-1)

P(α,β/y) α  (α+β)^-(5/2) *  ∏   

The comparison between the posterior distribution of  the parameter theta and thr original proportions can be made here:

It can be seen from the graph that the original values and the theta values are close to each other because the points and the line is close. It can be said that the distribution is a good fit.

(d)

For popn 1:

The expected value of the beta distribution with parameters   will be given by

Therefore,  expected value is 0.4086.

R codes for calculation :

p = 0.152/(0.152+0.220)

c1 = c(16/58, 9/90, 10/48, 13/57, 19/103,20/57, 18/86, 17/112, 35/273, 55/64)

quantile (p,c1)

(e)

The question demands for a prediction from posterior distribution. This can be made here by generating samples or generating new  from the beta(, that is

 ~ beta(

With the use of these sampled values, another sampling can be made from binomial distribution with a sample size of 100, that s n = 100.

y/ ~ Bin(100,

R cdes for simulation and confidence interval estimation are:

theta_new  = rbeta(S,alpha,beta)

Y_new = rbinom(S,100,theta_new)

Quantile(y_new,c(0.152,0.220))

(f)

It can be seen that the plot between   and y/n fits each other well. Therefore, it can be concluded  here that   has reasonable values.

References

Benavoli, A., Corani, G., Demšar, J. and Zaffalon, M., 2017. Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis. The Journal of Machine Learning Research, 18(1), pp.2653-2688.

Moore, B.R., Höhna, S., May, M.R., Rannala, B. and Huelsenbeck, J.P., 2016. Critically evaluating the theory and performance of Bayesian analysis of macroevolutionary mixtures. Proceedings of the National Academy of Sciences, 113(34), pp.9569-9574.