handle clicks outside inventory and non modifying clicks

handle clicks outside inventory and non modifying clicks #2110