About 6 months ago I made this rigid body animation https://www.dropbox.com/s/903733a7geqocwb/Vortex.mp4?dl=0 but one thing has been bugging me If you have a look at the animation at times the spheres do not collide but go through each other?
Here is a screenshot of my rigid body settings for the spheres
not too sure what it could be when I was doing it I noticed the problem read on a few forums online how I should set to higher sub-steps and solver iterations in the scene properties to increase the quality of the simulation but It did not do much
Have you tried changing the margin and also changing the collision shape?
Try using Mesh as the shape, and the source as Final.
Without seeing the file I can't say for sure what the issue is.
It looks like you have different rigid body setting for each sphere.
Best option is to set the settings on one and then duplicate the sphere, keeping the same settings for each.
If you want you can then tweak individuals slightly, bounciness for example.
Also you have keyframes on the selected sphere!!
What are these for? If they are location keyframes, this will cause issues.
If they are keyframes for the animated property, this is usually only used to start the physics process at a specific time in a larger animation, for this I wouldn't use it.
I would use two or three cameras and switch camera to the desired shot during the physics process.
Might be the margin I think that it might have been too small.
There is an option in Option > Rigid body to copy all the settings from active so all the settings are the same on the sphere.
There are 3 cameras being used and on them, I change the location, rotation, and focal length of the lens.
About a year ago I did a tutorial from another site on rigid bodies that said after baking the cache a few times if you are happy with the results bake the rigid body simulation to keyframes.