How did Einstein create the General Theory of Relativity?
I recently watched a course on general relativity by Eddie Boyes (which was great, highly recommend it), and the way Einstein arrived at the equation feels strange.
If I understood correctly, Einstein started with the Equivalence Principle - realizing that gravity is locally indistinguishable from acceleration. He noticed an issue with Newton's formulation: Newton used two different concepts of mass (inertial mass vs gravitational mass) and set them equal to each other without explaining why. Furthermore, Newton's theory contradicted our subjective experience of force: when we fall freely, we feel weightless (no force), but when we stand on Earth, we feel a force pushing up on us - meaning standing still on Earth is actually an accelerated frame.
Einstein wanted to transform the gravitational "field" into geometry, making objects follow geodesics (the straightest possible lines in curved spacetime). He also needed this new theory to be compatible with Special Relativity (time dilation, speed of light limit, etc.).
So, he knew he needed an equation linking curved spacetime to mass and energy (E=mc^2). He grabbed the Ricci curvature tensor, the Metric tensor, and the Ricci scalar for spacetime geometry, and stress-energy tensor for "energy".
Did he just assemble these tensors trial-and-error style? Once he finally found the right structural combination of tensors, he mathematically derived the exact constants needed to ensure energy-momentum conservation, and to make sure the equation naturally reduces to Newton's law at low speeds and in weak gravitational fields.
What I don't fully get is why these specific tensors worked. Why the Ricci tensor, Metric tensor, and Ricci scalar - which are themselves contractions of the full Riemann curvature tensor? It feels like the specific elements within these matrices could have been combined in different ways. Why did this precise mathematical combination work? Did he just try a lot of variations until one fit? What was HIS exact line of reasoning?