How do you decide whether an observed property of matter is a physical or a chemical property?

By the object's composition and the acting agent to which factors the change of the substance or entity.
The effect of physical property of matter on the object or substance can be better identified when the object doesn't change in composition or in nature. Unlike chemical property which has changed in its composition and atomic structure that was caused by chemical change or reaction due to an agent. Physical property is identified thru physical reactions or changes that has never changed the object in an atomic level, like cutting paper into smaller sizes. The aforementioned example illustrates physical proerty but being cut into smaller buts without changing the object from paper to any other substance, thus, its structure remain and its still called paper regardless of size, mass and texture. 

A.