How do I get rid of the graininess?

Two things I know you can do:

1. Turn on Denoising (Render Layers Properties window) and
2. increase the light level.

The main reason you get these sparkly bits (if memory serves) is because of low light conditions with high contrast colors.

@rontarrant Increasing the amount of samples also helps sometimes.

You can also turn on/off the caustics. In this case it will look less realistic however since there's a lot of glass and transparent objects.

Another trick is to clamp indirect lighting.