Information theoretical musings inspired by ARC-AGI

The extreme overfitting case of training is to have a map where each input leads to one output.

However it is cool that this overfit does not allow you to compute the final input for which there is no known output.

This therefore forces the creation of more general solution rules.

While in some cases solutions can work for any input, in many others they require specific assumptions about input, but the model could simply check that the assumptions apply to all inputs and use them for the final algorithm.

Bibliography: