When using booleans it's always a finicky situation. Just moving the boolean a bit can make the result jump in an instant and move it a little more makes it go away, so you have to find the point that gives you a good result. Also changing the solver works wonder. Try both options and see what happens.
