SUMMARY Some machine learning problems are more difficult than the others. Even if you have the most powerful machines and a billion instances to learn from, you might not be able to solve them. It is because the difficulty of the problems lies in the fact that the problems are born out of complicated and changing processes with some unobserved variables. In this note, we will present a solution framework for such cases. Let’s take the auto insurance sector as an example. For an auto insurance company, a key metric is the loss ratio (incurred losses divided by collected premiums). It depends on a number of directly observable factors such as the vehicle and driver type and their history, the local traffic, and even the general state of the economy. Insurance policies are then priced by the experts based on the predicted loss ratio for a certain future time period. The company employs insurance pricing experts who keep track of the variations in a limited number of key variables that affect the loss ratio. When they observe changes in these variables, they reflect those changes in the policy prices for certain micro-segments so that the insurance company realizes as little loss as possible. When predicting the loss ratio via machine learning, the variables that affect the future loss ratio are used as inputs. Particular effect of some of these variables on the loss ratio might be easy to predict. For example due to traffic patterns on a certain intersection, one can predict there might be more accidents there.[Should the insurance company report these findings to the transportation authorities and its customers? This is the subject for another article on machine learning and ethical responsibility.] On the other hand, the effect of the economy is very difficult to predict. Auto insurance policies are priced for a 12 month period and it is very difficult to predict an economic indicator such as the stock market or currency prices so far in the future, even if you use deep neural networks [Gunduz2017]. However, the human insurance pricing expert might have an idea on the fundamental factors that shape the economic conditions for his/her customer segments and make pricing decisions based on those. In short, valuable human expertise exists that can assist in future predictions.