Regarding the training of an unstable structure

[mm21d013]

I had a small query regarding M3GNet. I wanted to know whether the prediction coming out of the M3GNet model works well only when we take other structures alongwith the structure which is most stable ie ....whose energy above the hull plot is 0eV/atom?

Recently, I observed this pattern with whatever potential was developed by me. The moment I take the structure of a material which has energy above the hull plot around 20meV/atom....the entire potential prediction in combination with other structure fails. Only when I take in combination with the most stable structure..I get some good results. If you have any idea in this direction...can you please shed light on it .