Assuming a constant force if the mass of an object decreases, the acceleration of an object will

The acceleration of an object will increase.

Step by step:

This is because by solving for acceleration we obtain:

This means that if you divide by a smaller number the output will be larger, and since Force is the same it would take a larger and larger acceleration to obtain the same force.  Try your own numbers on a calculator to prove it to yourself.