An ideal omnidirectional camera is one that has spherical view that has single view-point, which means that just the sphere represents the directions, the bundle of directions, that are of interest and all directions converge on the centre. All the entire set of spherical directions is imaged at the same time, you do not have to do it sequentially. You have high resolution everywhere. And this happens without any distortions.
There are different approaches to omnidirectional viewing, there is the idea of packing many cameras such that they are centrally looking out of centre of the sphere physically. You pack many cameras physically, so that they are all really coming out of the centre looking in different directions.
So that is a very simple thing to do because you have many cameras and you can make them look in different directions now it turns out that doing such a thing is very difficult. If you do not want to leave gaps between cameras visual fields and you want to focus at the same time, so all of these things are not possible with this obvious natural thing. So you have to go some other route. An one of the obvious things to do is to pick a single camera instead of having many cameras looking in different directions at the same time, what you do is to take a single camera and let it rotate around the centre looking at different directions at different times. So after it has scanned the entire set of directions it can come back and redo the whole thing.
Another alternative is to use a single camera not multiple cameras, not a rotating single camera, but a single fixed camera, but have the lens such that it gathers light from very large angle, a very large cone so that its visual is very broad, which is what fish-eye lenses are meant for It is like the eye of a fish, it hat really very broad visual field, so the lens has roughly not quite but roughly the entire hemisphere visible to it. But it comes at a cost, so you have lots of distortion and variability of view-points on different directions.
Similarly, instead of having fish-eye lens to collect light from different directions an alternative that has been used is to put a mirror and gather the light from a very large number of directions, a emisphere or whatever angle, to let the mirror collect the light instead of the fish-eye lens. So these two are sort of analogues, dual of each other.