As always, check out the question.
A simple explanation for why surface brightness cannot increase is that it would violate the second law of thermodynamics. Instead of calculating, I’ll try to convince you of this with a gedankenexperiment.
Imagine a universe with one large ideal black body, one small “othello disk” (perfectly black on one side, perfectly white on the other), a parabolic reflecting mirror, and some dark matter to cause gravitational lensing. (In the picture below, the black body is orange, the mirror is gray, the disk is green, and optical paths are dotted black.)
The time scale on which the large black body comes to thermal equilibrium with its surroundings by radiation is much longer than the equivalent time scale for the disk, so throughout the course of the experiment we can consider the large black body to have a constant temperature. The disk is at the focus of the parabolic mirror. It is small enough and placed close enough to the large black body that the image of the large black body covers the surface of the disk. The mirror itself is smaller than the typical spatial scale of variations in the large black body’s radiation’s surface brightness (assuming it has some).
If there were no variations in surface brightness, the black side of the disk would be completely covered by the image of the large black body. Then the disk would come to equilibrium with the radiation coming in, and would reach the same temperature as the large black body.
Now imagine there are variations in surface brightness. Conservation of energy requires that if surface brightness decreases somewhere, it must increase somewhere else. So place the dark matter so that the mirror is at a patch of high surface brightness. Then the disk is still covered by the image of the black body, but that image is now brighter. When the disk comes to equilibrium, it must be hotter than the large black body, which was itself the source of the heat. So we have heat flowing from a cold body to a hot one, in violation of the second law. By R.A.A, the variations in surface brightness must not exist.