The adapter clicks on the lens. It has a small dark gray colored lever (not the white plastic tab to change aperture), level to the outer edge of the bottom ring accessible when removed from the camera, grab the whole adapter with your left hand and push the lever in with your thumb, and while pushing, rotate the lens with your right hand to remove it. You need to push in that lever to unlock the lens.

Yes, like Ishtar said, make sure you are pressing the button in. It takes a fair amount of pressure and you need to press it in pretty deep. Holding the lens while turning the adapter AND pressing in the button in continuously can be a coordination challenge so maybe have a friend hold the lens while you push and turn? :-)

Make sure you press the button while releasing it.