April 5, 2009

model      BIC           deltaBIC            AIC          deltaAIC
1       141978.9     112.05478     141789.5     154.048607
2       141924.9     57.98382       141648.5     13.137065
3       142006.4     139.49803     141643.2     7.810695
4       141866.9     0.00000         141677.4     41.993823
5       141911.7     44.84676       141635.4     0.000000

n ~ 39,000

The AIC seems to always select model 5 when examining all the data or based on random splits (for model cross-validation). However, the BIC will select model 4 (which is more parsimonious than model 5) on random splits of the data but will select model 5 on all the data. Which model selection criterion to go with?


One comment

  1. I got that kind of inconsistency in my own research a lot. In that case, I would also conduct a likelihood ratio test between the two and see if the more parsimonious one yields significantly differ from the less parsimonious one. Of course, these two models must be nested.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: