Would like to write a geometry node to stack the books with different size in a collections and stack them layer by layers without overlapping each other and with some random rotation in Z-axis.
No idea how to get this result.
Look around for some add-on to do but in vein.

I'm not sure how to go about it with Geo Nodes, but you can always do it the old way, manually and artistically placed by hand. It wouldn't take much time with a couple of arrays and then applying them and go in and moving them. Put a little bit of music to pass the time, may I suggest Oasis, Definitely Maybe album.
And if you don't want to do it by hand, maybe you should have a look at this, to get an idea, how this can be done:
https://www.youtube.com/watch?v=rLfxq5KavzM&t=5s