Unlimited Detail is interesting, but I have very specific doubts ...
A few days ago, Hacker News video (and thanks ayambit for reminding about them on Habré) skipped about the company Unlimited Detail , which promises to make the most advanced NVidia and ATI cards useless and render scenes in billions of points on a conventional processor (CPU), without 3D acceleration (!) at all. Exceeding the capabilities of modern video cards hundreds of times (thousands, according to the authors, which I will dispute below).
If anyone has not seen:
')
First, just in case, about what it is, and then that I, as a programmer, have doubts that on netbooks without 3D cards, they will soon be playing Crysis 3 ...
(further a lot of words and technical details)
So, being fascinated by all sorts of algorithms of graphics and computer vision and being quite familiar with OpenGL - I was really interested in all this. What is told in the video? What is the usual graphics, as it is now - consists of polygons - flat surfaces. The better your card, the more polygons it can draw. But real objects are often rounded, as a result, being built from a limited set of polygons - they seem angular (examples with pots, hatches, stones and grass in the video) and although there are methods to circumvent in some sense these limitations - they have their own problems .
One of the methods for circumventing a polygon plane constraint is parallax mapping. About him say the developers in the video.
The problem is obvious - the edge still remains a straight line. Plus it gives a big load (due to the calculations of shaders) on the video card, so you don’t draw on them.
The second option is tesselation. This is a much more advanced method from the field of DirectX 11 (and OpenGL 3, which already has version 4 by the way) - but here the video is worth hundreds of words:
As you can see - the stones “honestly” protrude at the corner. True demo (if memory serves) for a high-end card -GeForce 28x ... under GF100 (GeForce GTX 4x0) and ATI R800 (Radeon 5xxx) (thanks infi for the amendment)
In general, tesselation on modern boards makes it possible to circumvent in many respects the limitations of polygons almost as much as Unlimited Detail (UD). This truth does not say the authors in the video.
Okay, back to our UD. So, the authors state that they have developed a search algorithm that allowed them to render scenes with billions (!) Points in real time. It is the point cloud that is rasterized (rendered), in other words, there are no triangles, there are only many, many, many small points.
For comparison, I somehow tested for a modern card (8600GTS) and it was able to render at 24 fps only 10 million points. UD claims billions (!), That is, 8600s need to buy 100 pieces. With that, the performance growth of video cards is only 22% per year (according to UD in video) - we will have to wait for a similar FPS from 3D accelerators for a long time. (23 years seems to be obtained? 1.22 ** 23 = 100)
What makes UD have speed? Due to the fact that you have a total of 1024x768 = 786000 pixels on the screen. If we know which particular 3D point - which one on a flat monitor corresponds to - then we need only 786,000 * 25 = 20 million operations per second. A modern processor can make up to 3 billion. So in principle, it is possible on the processor. But what a thing - so far no one has really found an algorithm to accurately calculate this minimum set of points.
In modern games, the so-called occlusion culling is used.
As you can see, when the grid is turned on - some of the houses just disappear (not drawn). At a more advanced level, culling works at the level of literally polygons - hiding all those that are not visible (closed by something, behind, out of sight, although the last two are slightly different and very old culling - frustum culling).
So - in fact, the authors of UD invented such an algorithm, but only for points. That is, it allows you to find the minimum number of points that you need to draw and only show them. The same occlusion culling, but at the level of pixels - which one will be closed.
It would seem - there is happiness! The grass will cease to be flat, all the branches will be visible, and even sleepers will become convex.
Well, comrades tuneya ... software engineers ... let's turn on the brains now and figure out what's wrong here ... (I have doubts) ...
First, as stated in the video, NVidia and ATI do not really crave this technology. It would seem to be understandable - who will need them with $ 1,000 cards every 2 years, when everyone will play on the CPU. But on the other hand, why not buy and not bury? Apparently not worried. And why?
So ... let's try to think. We need 20 million operations in order to play 1024x768 at 25 fps. The modern percents do 3 billion operations, it means that no more than 150 lower-level teams should go to _search_ and drawing one point. All right, let's say rendering is essentially a multiplication by a 4x4 matrix and there is also division. We assume that this is possible, but we come to the search.
So, 2 billion pixels, as they say, among which you need to look ...
We do not have some popular algorithms for compressing point clouds (as for pictures - JPG, DDS, etc.), it means that all points are stored in memory. Then we talk about the scene in the jungle, where there are few repetitive elements. 2 billion dots * (4 bytes (float) * 3 coordinates + 3 colors * 1 byte (uint8)) = 30 gigabytes of RAM ... so, we obviously do not get into the home computer ...
Well, for quite complete paranoia: notice that the scene is in the jungle (2 billion points, 30 gig in theory) - it is maaaaalusenkaya compared to any level in the same Crysis, which can be more than 100 times. So the theoretical 30 gigs can be multiplied by 100 ... or we will wait for each output from each field to load new 30 gigs of new field (at a hard drive speed of 30MB per second = 1000 seconds or 16 minutes per field meadow). :) As in the good old days of the very first games - 10 steps ... "loading ...."
What are the options? Winchesters - they are big, 30 gigs is not a question ... They build some kind of index and keep it in memory, 30 gig data is stored on the hard drive, and they are addressed as needed ... then I have questions. Any access to the hard drive (random seek) is 9ms - that is, just a hundred readings per second. That is, no more than 4 searches for one frame.
And if you look at the video - then random seek (reading out of order) will be required anyway, because some "elephants" are close to the camera, others in FIG know what distance. That is, any movement is slightly to the left a little to the right and we need to re-read even if not a very large number (768,000 points at most), but points scattered over the 30gig file. If these points were all written down in a row - there is no problem - dozens of megs per second are read in order by second, but random reading is still in trouble ... and we only have 4 searches per frame (760 thousand points in 4 request essentially need to read).
So here we are stuck in two questions - where it is all stored and how it is reproduced ... most likely developers use machines with gigs of operatives and data all in memory. But we have dozens of gigs with you.
Of course, I remember Elite for the Spectrum almost the limitless universe crammed into 64Kb, andtheprodukkt Fabrausch pushed the whole city into 181,000 bytes ( very cleverly .. in a smart way ). But something tells me that the method of procedural textures will not really save the authors here.
Groundwork All technologies so far have been designed for landfills. All 3D editors (with the exception of specific ones, with which I worked, but even their names will not tell anyone) are designed for polygons. All 3d artists think of polygons (of course, zBrush changes the game scene a bit, but all the same, initially the work comes with a low-poly model). All texture makers are used to working with polygonal models ... all model optimization algorithms are designed for polygons ... all compression algorithms are designed for flat textures (JPG, DDS).
There simply are not enough tools for processing point clouds, or people who can draw in them, or compression algorithms. There is a small scene shown in the video, which, aswe found out, I assume that above it will take (only it!) 30 gigs of space ... and there is no special compression algorithm for this kind of data!
Of course, the developers say that you can take regular models and transfer them (with textures) to the point cloud, but they admit that they have not done so yet. This may solve the problem of the absence of artists, but it does not solve the problem of compression and volume.
Developers say that “do not pay attention to flickering shadows - we did not smooth out the shadow map” (basically, we are talking about the shadow mapping parameters - this is understandable, they need to be tuned for specific scenes), but the problem still exists. In particular, almost everything that impresses you in modern games is shaders. Any highlight is the result of the shader. And with shaders, one of the main inputs for it is the normal point on the surface. ( picture ) This arrow (only at each point of the triangle) is what determines how strong the “flare” in it is. It is from her that the same displacement mapping, reflections (especially!) And the refraction of light, dance.
Now the problem. At a point in a point cloud there is NO NORMAL ... there is no place for her to hold the arrow - it is just a point. For a normal, it needs to know at least two other points with which it will form a surface (well, or the direction of the normal). Therefore, it cannot reflect, refract, gleam, etc. Simply put, a true point cloud cannot even be a glossy plastic (or even a not perfectly matte plastic) - glare is excluded - they simply cannot be calculated. Therefore, everything that you see on the video - in general, frosted - no glare.
In fact, here I am lying a little - and of course if you use the old methods of polygonal modeling - points can have normals. But in the honest cloud there are no normal points . Normals, of course, can be, you just need to take them from somewhere, and for this you need to have triangles or polygons.
This is an example of what is generally a point cloud, which render Unlimited Detail. By the way, it should be noted that this explains that the detailing is not “infinite” there. It is finite, the authors had in mind that they can search for a level almost unlimited in size (which is, in general, doubtful).
The retreat why the shadows work: With the shadows everything is made easier here - the scene is rendered again from the point of view of the sun (invisible) and which of these points (“visible” to the sun) are compared are visible and to the observer in the camera - those that are visible - bright ( in the sun), those that are not visible - in the shade.
Of course, in the video we see a reflection (a la Quake 1 :)) in the lake. But the developers admit that they cheated and in fact the scene is double - one up, the other down. True, such a primitive effect (reflection from a plane) could be mathematically easy to calculate. Another question is the familiar “ripples” ... here it is not so easy to calculate the processor - time is needed for ray tracing ...
Therefore, all the classy effects like glare, reflections will have to be temporarily abandoned (until they figure out how to calculate point normals or between points in real time in a cloud of points) and do all the games in cloudy weather (a la Doum, although it seems there were already reflections ...)
Simply put, we look here - only the first two options are without normals and possible:
Another problem is transparency . If UD is looking only for the NEAR pixel to the camera that will be visible - then finding which ones behind this nearest one may be a problem not provided for by the search algorithm. Namely, this is how transparency works - 50% of the front pixel and mixed with the colors of those behind.
4. Will a
One of the main problems now in the gaming industry is that you have to do objects that are too detailed (I’m against it - give me a fun, lively game with moderate graphics than a more realistic game ... if I wanted reality, I’d go to the street , and not for the game) It is because of this that tens of millions of dollars are already being spent on games ... GTA4 - $ 100 million !
So, now the corners are still cut ... although not, on the contrary, just because the games still have corners in 3D models - the price is still moderate for the development of objects. And what if each item of the game will have to be developed in zBrush ... by points ... to millimeters detailing ... the prices for games will jump even higher ... of course, Russia doesn’t care much, but nevertheless is another reason that the gaming industry resists the introduction of such technologies as unlimited detail.
I completely forgot to say about it too ... now the video cards are being rescued by developers by leaving the percentages almost free to calculate AI and physics (although the latter is also shifted to the video cards). In the case if the percent will be engaged only in the calculation of graphics - AI and physics will have no one to count. As well as the so-called skinning - bending the character model during the animation with the help of bones. This is also now either a CPU or a GPU is engaged. But everything will have to be on the CPU, which will already be busy.
So maybe for the "infinite detail" without a video card, you will have to pay stupefied AI and obsolete players passing through the ground due to the lack of CPU time ...
Of course, multiprocessing can help out, but so far in the current realities there is one more little nail in the foot of Unlimited Detail.
In general, the video is impressive. And I have no doubt that the authors did what they show. Maybe somewhere cut corners (like 32gb operatives), but still it exists. Moreover, hackers from Hacker News have unearthed that the authors of UD may be among the leading developers in the world in the field of algorithms (Greg Douglas, perhaps the same as that related to the development of the R-Tree algorithm) and have published works ... but here's something (after all), I doubt that it will be allowed into the mainstream gaming ... (at least not in the next 16 months, as Unlimited Detail has developed).
But, as in any good business activity - the more people say that what you are doing is impossible, the better way (perhaps) you have found. So maybe the gentlemen from UD have found how to get around all these problems and this technology can really work on laptops ... But so far, even in theory, I cannot sweep aside one of them (... maybe only about normals ...) and IT -expert suggests that miracles do not usually exist.
And more importantly, the gaming industry has no such incentive to take this technology and redo all of its tools for it, retrain people and develop new algorithms for compressing and storing such data, when with the approaching level of video cards, the detail is almost indistinguishable from what UD offers. And visually, the UD demos lose almost every modern game in 7 years.
Predicting the course of history is a thankless task, you never guess where it will turn. I just voiced the theoretical problems that I see.
Well, time will tell ...
Yoi Haji
view from Habra
PS It turns out Unlimited Detail found their way 2 years ago ...
If anyone has not seen:
')
First, just in case, about what it is, and then that I, as a programmer, have doubts that on netbooks without 3D cards, they will soon be playing Crysis 3 ...
(further a lot of words and technical details)
Unlimited Detail
So, being fascinated by all sorts of algorithms of graphics and computer vision and being quite familiar with OpenGL - I was really interested in all this. What is told in the video? What is the usual graphics, as it is now - consists of polygons - flat surfaces. The better your card, the more polygons it can draw. But real objects are often rounded, as a result, being built from a limited set of polygons - they seem angular (examples with pots, hatches, stones and grass in the video) and although there are methods to circumvent in some sense these limitations - they have their own problems .
In the old-fashioned way, using OpenGL 3 / DirectX11 methods - parallax mapping, tesselation
One of the methods for circumventing a polygon plane constraint is parallax mapping. About him say the developers in the video.
The problem is obvious - the edge still remains a straight line. Plus it gives a big load (due to the calculations of shaders) on the video card, so you don’t draw on them.
The second option is tesselation. This is a much more advanced method from the field of DirectX 11 (and OpenGL 3, which already has version 4 by the way) - but here the video is worth hundreds of words:
As you can see - the stones “honestly” protrude at the corner. True demo (if memory serves) for a high-end card -
In general, tesselation on modern boards makes it possible to circumvent in many respects the limitations of polygons almost as much as Unlimited Detail (UD). This truth does not say the authors in the video.
What did unlimited detail do?
Okay, back to our UD. So, the authors state that they have developed a search algorithm that allowed them to render scenes with billions (!) Points in real time. It is the point cloud that is rasterized (rendered), in other words, there are no triangles, there are only many, many, many small points.
For comparison, I somehow tested for a modern card (8600GTS) and it was able to render at 24 fps only 10 million points. UD claims billions (!), That is, 8600s need to buy 100 pieces. With that, the performance growth of video cards is only 22% per year (according to UD in video) - we will have to wait for a similar FPS from 3D accelerators for a long time. (23 years seems to be obtained? 1.22 ** 23 = 100)
What makes UD have speed? Due to the fact that you have a total of 1024x768 = 786000 pixels on the screen. If we know which particular 3D point - which one on a flat monitor corresponds to - then we need only 786,000 * 25 = 20 million operations per second. A modern processor can make up to 3 billion. So in principle, it is possible on the processor. But what a thing - so far no one has really found an algorithm to accurately calculate this minimum set of points.
In modern games, the so-called occlusion culling is used.
As you can see, when the grid is turned on - some of the houses just disappear (not drawn). At a more advanced level, culling works at the level of literally polygons - hiding all those that are not visible (closed by something, behind, out of sight, although the last two are slightly different and very old culling - frustum culling).
So - in fact, the authors of UD invented such an algorithm, but only for points. That is, it allows you to find the minimum number of points that you need to draw and only show them. The same occlusion culling, but at the level of pixels - which one will be closed.
It would seem - there is happiness! The grass will cease to be flat, all the branches will be visible, and even sleepers will become convex.
Well, comrades tuneya ... software engineers ... let's turn on the brains now and figure out what's wrong here ... (I have doubts) ...
Something is wrong here...
First, as stated in the video, NVidia and ATI do not really crave this technology. It would seem to be understandable - who will need them with $ 1,000 cards every 2 years, when everyone will play on the CPU. But on the other hand, why not buy and not bury? Apparently not worried. And why?
So ... let's try to think. We need 20 million operations in order to play 1024x768 at 25 fps. The modern percents do 3 billion operations, it means that no more than 150 lower-level teams should go to _search_ and drawing one point. All right, let's say rendering is essentially a multiplication by a 4x4 matrix and there is also division. We assume that this is possible, but we come to the search.
So, 2 billion pixels, as they say, among which you need to look ...
1. Where have you stuffed all this?
We do not have some popular algorithms for compressing point clouds (as for pictures - JPG, DDS, etc.), it means that all points are stored in memory. Then we talk about the scene in the jungle, where there are few repetitive elements. 2 billion dots * (4 bytes (float) * 3 coordinates + 3 colors * 1 byte (uint8)) = 30 gigabytes of RAM ... so, we obviously do not get into the home computer ...
Well, for quite complete paranoia: notice that the scene is in the jungle (2 billion points, 30 gig in theory) - it is maaaaalusenkaya compared to any level in the same Crysis, which can be more than 100 times. So the theoretical 30 gigs can be multiplied by 100 ... or we will wait for each output from each field to load new 30 gigs of new field (at a hard drive speed of 30MB per second = 1000 seconds or 16 minutes per field meadow). :) As in the good old days of the very first games - 10 steps ... "loading ...."
What are the options? Winchesters - they are big, 30 gigs is not a question ... They build some kind of index and keep it in memory, 30 gig data is stored on the hard drive, and they are addressed as needed ... then I have questions. Any access to the hard drive (random seek) is 9ms - that is, just a hundred readings per second. That is, no more than 4 searches for one frame.
And if you look at the video - then random seek (reading out of order) will be required anyway, because some "elephants" are close to the camera, others in FIG know what distance. That is, any movement is slightly to the left a little to the right and we need to re-read even if not a very large number (768,000 points at most), but points scattered over the 30gig file. If these points were all written down in a row - there is no problem - dozens of megs per second are read in order by second, but random reading is still in trouble ... and we only have 4 searches per frame (760 thousand points in 4 request essentially need to read).
So here we are stuck in two questions - where it is all stored and how it is reproduced ... most likely developers use machines with gigs of operatives and data all in memory. But we have dozens of gigs with you.
Of course, I remember Elite for the Spectrum almost the limitless universe crammed into 64Kb, and
2. Developments, 3D artists, unemployment, riots ...
Groundwork All technologies so far have been designed for landfills. All 3D editors (with the exception of specific ones, with which I worked, but even their names will not tell anyone) are designed for polygons. All 3d artists think of polygons (of course, zBrush changes the game scene a bit, but all the same, initially the work comes with a low-poly model). All texture makers are used to working with polygonal models ... all model optimization algorithms are designed for polygons ... all compression algorithms are designed for flat textures (JPG, DDS).
There simply are not enough tools for processing point clouds, or people who can draw in them, or compression algorithms. There is a small scene shown in the video, which, as
Of course, the developers say that you can take regular models and transfer them (with textures) to the point cloud, but they admit that they have not done so yet. This may solve the problem of the absence of artists, but it does not solve the problem of compression and volume.
3. We do not need any glare and reflections, we will play in Doom!
Developers say that “do not pay attention to flickering shadows - we did not smooth out the shadow map” (basically, we are talking about the shadow mapping parameters - this is understandable, they need to be tuned for specific scenes), but the problem still exists. In particular, almost everything that impresses you in modern games is shaders. Any highlight is the result of the shader. And with shaders, one of the main inputs for it is the normal point on the surface. ( picture ) This arrow (only at each point of the triangle) is what determines how strong the “flare” in it is. It is from her that the same displacement mapping, reflections (especially!) And the refraction of light, dance.
Now the problem. At a point in a point cloud there is NO NORMAL ... there is no place for her to hold the arrow - it is just a point. For a normal, it needs to know at least two other points with which it will form a surface (well, or the direction of the normal). Therefore, it cannot reflect, refract, gleam, etc. Simply put, a true point cloud cannot even be a glossy plastic (or even a not perfectly matte plastic) - glare is excluded - they simply cannot be calculated. Therefore, everything that you see on the video - in general, frosted - no glare.
In fact, here I am lying a little - and of course if you use the old methods of polygonal modeling - points can have normals. But in the honest cloud there are no normal points . Normals, of course, can be, you just need to take them from somewhere, and for this you need to have triangles or polygons.
This is an example of what is generally a point cloud, which render Unlimited Detail. By the way, it should be noted that this explains that the detailing is not “infinite” there. It is finite, the authors had in mind that they can search for a level almost unlimited in size (which is, in general, doubtful).
The retreat why the shadows work: With the shadows everything is made easier here - the scene is rendered again from the point of view of the sun (invisible) and which of these points (“visible” to the sun) are compared are visible and to the observer in the camera - those that are visible - bright ( in the sun), those that are not visible - in the shade.
Of course, in the video we see a reflection (a la Quake 1 :)) in the lake. But the developers admit that they cheated and in fact the scene is double - one up, the other down. True, such a primitive effect (reflection from a plane) could be mathematically easy to calculate. Another question is the familiar “ripples” ... here it is not so easy to calculate the processor - time is needed for ray tracing ...
Therefore, all the classy effects like glare, reflections will have to be temporarily abandoned (until they figure out how to calculate point normals or between points in real time in a cloud of points) and do all the games in cloudy weather (a la Doum, although it seems there were already reflections ...)
Simply put, we look here - only the first two options are without normals and possible:
Another problem is transparency . If UD is looking only for the NEAR pixel to the camera that will be visible - then finding which ones behind this nearest one may be a problem not provided for by the search algorithm. Namely, this is how transparency works - 50% of the front pixel and mixed with the colors of those behind.
4. Will a Russian American pay $ 500 per torre game?
One of the main problems now in the gaming industry is that you have to do objects that are too detailed (I’m against it - give me a fun, lively game with moderate graphics than a more realistic game ... if I wanted reality, I’d go to the street , and not for the game) It is because of this that tens of millions of dollars are already being spent on games ... GTA4 - $ 100 million !
So, now the corners are still cut ... although not, on the contrary, just because the games still have corners in 3D models - the price is still moderate for the development of objects. And what if each item of the game will have to be developed in zBrush ... by points ... to millimeters detailing ... the prices for games will jump even higher ... of course, Russia doesn’t care much, but nevertheless is another reason that the gaming industry resists the introduction of such technologies as unlimited detail.
5. AI? Physics? Bones?
I completely forgot to say about it too ... now the video cards are being rescued by developers by leaving the percentages almost free to calculate AI and physics (although the latter is also shifted to the video cards). In the case if the percent will be engaged only in the calculation of graphics - AI and physics will have no one to count. As well as the so-called skinning - bending the character model during the animation with the help of bones. This is also now either a CPU or a GPU is engaged. But everything will have to be on the CPU, which will already be busy.
So maybe for the "
Of course, multiprocessing can help out, but so far in the current realities there is one more little nail in the foot of Unlimited Detail.
Generally...
In general, the video is impressive. And I have no doubt that the authors did what they show. Maybe somewhere cut corners (like 32gb operatives), but still it exists. Moreover, hackers from Hacker News have unearthed that the authors of UD may be among the leading developers in the world in the field of algorithms (Greg Douglas, perhaps the same as that related to the development of the R-Tree algorithm) and have published works ... but here's something (after all), I doubt that it will be allowed into the mainstream gaming ... (at least not in the next 16 months, as Unlimited Detail has developed).
But, as in any good business activity - the more people say that what you are doing is impossible, the better way (perhaps) you have found. So maybe the gentlemen from UD have found how to get around all these problems and this technology can really work on laptops ... But so far, even in theory, I cannot sweep aside one of them (... maybe only about normals ...) and IT -expert suggests that miracles do not usually exist.
And more importantly, the gaming industry has no such incentive to take this technology and redo all of its tools for it, retrain people and develop new algorithms for compressing and storing such data, when with the approaching level of video cards, the detail is almost indistinguishable from what UD offers. And visually, the UD demos lose almost every modern game in 7 years.
Predicting the course of history is a thankless task, you never guess where it will turn. I just voiced the theoretical problems that I see.
Well, time will tell ...
Yoi Haji
view from Habra
PS It turns out Unlimited Detail found their way 2 years ago ...
Source: https://habr.com/ru/post/87208/
All Articles