VGMaps
General Boards => Map Gab => Topic started by: JonLeung on May 27, 2007, 09:07:11 pm
-
Today I posted sprays's Circle Of The Moon map, which is the largest map we've seen yet - a 16.1 MB .PNG!
I noticed that the Harmony Of Dissonance maps (by Zeric) were put up on the 28th of last month - almost exactly a month ago - and those come in at 11.1 MB and 10.4 MB. A week ago I put up The Faery Tale Adventure map by DarkWolf, an 8.48 MB .png (the map isn't even full scale, wow), and that rounds out the top 4 largest maps, beating out the previous record holder, which was a .JPG (The Citadel Island map from Relentless: Twinsen's Adventure, made by Assassin, 7.23 MB) so the different filetypes don't really make it a fair comparison.
Of course, there's the issue of .PNG compression since we're talking about filesize... sprays thought the advancecomp program was a better compressor than PNGGauntlet (which itself is essentially PNGOut, I believe). If PNGGauntlet/PNGOut was used on the Circle Of the Moon map and truly didn't compress it as much, it doesn't change its heavyweight status as it would still be the largest. I guess the question is how large The Faery Tale Adventure's map would be if it was full-sized?
As I said in the news post, the sudden flow of large maps is interesting. Is there some kind of competition going on to see who can make the largest map? o_0
If there were such things as animated .PNGs (or animated .GIFs that could somehow have 16.7 million colours instead of just 256 maximum), a full-sized, pixel-perfect, animated map of Zebes from Super Metroid would probably be pretty large... >_>
-
The true size of the Faery Tale Adventure World map is 128 x 128 tiles. (I trimmed off a few of the water tiles on the 1:4 map to make it look a little more centered.)
Each tile is 256 x 256 in size.
(256 * 128)^2 = 1,073,741,824 pixels OR a 32,768 x 32,768 image.
To display the image in 24-bit color the computer will need three bytes for each pixel in memory. So the final result is 3,221,225,472 bytes, or exactly 3 gigs of memory (if you use 1024 instead of 1000).
Of course, an 8-bit PNG would be less space on disk, but viewing the image would still require loads of memory.
Doing a full size PNG is out of the question. Even if I wrote a C program to write the image to disk tile by tile, who'd be able to open it?
I can do an HTML table version of the map which seems to work fairly well in most browsers. If someone wants to really see a 1:1 representation I'll do it.
-
The world maps of The 7th Saga I've done (Ticondera - past and present), if left whole, would have been 17408 x 13376 and about 10 MB (un-PNGGauntleted) each.
-
While we are on the subject of large maps, what program(s) do you recommend to view those monsters? Is there a Windows program or IE/Firefox addon that can handle them? I am trying to view the Metroid 2 overworld map, and I cannot find a program that can process the file. (And the map is only 3 MB.)
Thanks.
-
The default Windows Picture and Fax Viewer has been able to handle any file size I've ever thrown at it.
-
I tried that but it usually stalls/crashed when I give it a big file.
-
I've always been a fan of Irfanview, a freeware graphic viewer that's small and easy to use. It can also be used to convert pictures to a large number of file formats, and resize them or alter their color depth, plus a whole lot of other options.
It's been my best friend for a while now, and I've always used it to browse through screenshots when pasting them in Paint.
---
"With the first link, the chain is forged. The first speech censored, the first thought forbidden, the first freedom denied, chains us all irrevocably." [...] The first time any man's freedom is trodden on, we're all damaged. - Captain Jean-Luc Picard
B*tch, meet reality. Reality, meet b*tch. - Me
-
Irfanview works great! Thanks Terra.
-
I use CompuPic Pro, which is handy for many things. I've ranted enough about it in other topics. It's certainly not freeware, though.
As for viewing things while still online, as I mentioned, Opera had no problem with the biggest maps that IE refused to load. I suppose there are limitations around your related configurations and hardware and memory, so you may have different results. I'm not an advocate of any browser in particular (I'm "guilty" of sticking with IE for the most part, but I certainly don't love Microsoft, I just stick with what I'm used to) but having other browsers may be one simple workaround.
-
I consider Internet Explorer the plague of web surfing. Kinda like sleeping with lots of old and sick prostitutes without any protection whatsoever. Anyway, as soon as install Windows anywhere, I immediately install Mozilla Firefox to minimize the risks of spyware, annoying popups and dangerous ActiveX code. But really, anything except IE will do that.
---
"With the first link, the chain is forged. The first speech censored, the first thought forbidden, the first freedom denied, chains us all irrevocably." [...] The first time any man's freedom is trodden on, we're all damaged. - Captain Jean-Luc Picard
B*tch, meet reality. Reality, meet b*tch. - Me
-
I could have sworn I had another more recent topic about large maps...
Anyway, in terms of filesize, Command&Conquer-RedAlert2-Yuri'sRevenge-Allies-ALL02UMD-Mission2-HollywoodAndVain.png (https://www.vgmaps.com/Atlas/PC/index.htm#CommandConquerRedAlert2) (25.6 MB) has been bested by RageOfMages-26-RoadToTheTower.png (https://www.vgmaps.com/Atlas/PC/index.htm#RageOfMages) (54.2 MB), which as you can see is more than double the filesize! LDK says he's PNGGauntleted that already, so I'll take his word for it because I know it'll take forever to run it myself. LDK sure loves his PC RTS games, which seem to easily pass the 15+ MB mark.
-
I ran it through PNGOUT with /f0 and saved 8MB.
Edit: and the C&C map you mentioned isn't optimised at all, it seems.
-
Every map I submit is procesed by PNGGauntlet. Simply because Photoshop has slow PNG export so I'm saving every map in BMP and then compress them by PNGGauntlet with Xtreme! option. Advanced options are set to default.
I've ran CnC map again and it didn't save a byte. But maybe I'm doing something wrong
-
As I've mentioned before, screenshot maps generally compress best with the /f0 parameter. PNGOUT defaults to /f0 for paletted images and /f5 for true-colour, so maps with more than 256 colours need to override this default. PNGGauntlet lets you do this by choosing "Filter Type: None" in the advanced options, but it's worth checking both settings. The "compression strategy" in PNGGauntlet is kind of pointless, there's never any reason to not use "Xtreme!".
That C&C map reduces to 14.5MB, such a huge change I assumed it was a poorly-compressed source image. However, at /f5 it does come out the same, so I'd assumed wrong.
-
You are right. I'm running CnC maps with None filter type and there is 30 - 40% size reduction. Kinda stupid not to read manual >:(
Jon do you want me to send you all my files again? (I will be recompressing them anyway)
-
It is possible to run PNGOUT directly on the files on the webserver, but the host might not be happy for you to run at 100% CPU for a few days (!).
-
Sure, LDK, I'll take any and all of your files again if you're going to be recompressing them.
I guess I should check my PNGGauntlet settings and see if I have to do another massive site recompression. >_< At the very least, I should redo the really big ones that could save a few MB.
Thank goodness with GoDaddy I don't ever run out of bandwidth like my previous host sometimes did, but it's not so much about bandwidth now but more so that users can view them as quickly and easily as possible.
-
Theoretically if you're running on a Linux server you could nice the pngout command to have a very low priority, but still, some hosts might find that annoying. And if you are on a shared host with lots of traffic it might not even be worth it.
Every once in a while I throw the idea around in my head of writing a distributed PNGout application. Not something on the scale of BOINC or other grid computing applications, but at least being able to setup a few computers that could do a batch of work in their spare time. If Jon wanted to re-compress the maps on the site, I would be willing to lend a hand (and some computing power).
-
Every map I submit is procesed by PNGGauntlet. Simply because Photoshop has slow PNG export so I'm saving every map in BMP and then compress them by PNGGauntlet with Xtreme! option. Advanced options are set to default.
I'm a little scared to use PNGGauntlet again, because the last time I did, it just took up more and more of the CPU time until it finally froze up my computer. ??? Why do you think that happened?
-
It's supposed to use up all the CPU, it's using it to compress the image. It'll use up memory too, which can be an issue for large images.
-
In my experience PNGGauntlet will sometimes act strangely on large files or large groups of files. For larger images it's just better to run pngout. Even for a directory of smaller images I find pngout is just easier from the command line. Also for smaller images I like to run pngout a few times with the /r switch after the first pass to get rid of a few additional bytes.
-
I'm currently working on a maps for Alien Shooter 2 which has pretty huge maps. So big that PNGOUT is refusing to compress them and exits with "malloc failed" error. My guess is that I'm hitting memory limitation of 32-bit. Unfortunatelly I can't find 64-bit PNGOUT for windows so I'm stuck. So any ideas how to bypass this error and compress these big files? Photoshop has very bad PNG output and PNG's from PS can be compressed to cca 60% of their original size.
one of the bigger levels from game: 18836 x 13674 pixels, uncopressed PNG: 273,6MB
(http://img842.imageshack.us/img842/1640/75373941.th.png) (http://img842.imageshack.us/i/75373941.png/)
-
I think there is a 64-bit version for Linux somewhere, but not for Windows. You could try increasing your virtual memory limit, but that won't help if the program itself is actually using more than 4 GB of RAM.
For these huge maps perhaps using the Google Maps API (or something similar) would be more appropriate?
-
Try JPG. See if it looks bad, and how the filesize compares. Otherwise you'll have to divide the map into smaller segments so that people can actually view it.
-
Other PNG optimisers might have 64-bit builds (if you have a 64-bit OS). Your example image is only (!) 0.9GB uncompressed (assuming the worst case of 4 bytes per pixel) so that ought not to be an issue, though. (Well, maybe compression takes up more memory than I'd expected.)
Ken Silverman (PNGOUT guy) on 64-bit builds (http://www.jonof.id.au/forum/index.php?topic=1515.msg13574#msg13574) - maybe you can post there and he'll be convinced it's needed. It'll take forever to run, though...
-
DarkWolf: Physical memory is not problem, I have 4GB RAM and dynamic swap file. Problem is 2GB limit on 32-bit processes. Google Maps API would be ideal but I don't have any experience with it. And I don't know if it could run on vgmaps it's realy not up to me.
Revned: Tried JPG on quality 10 and get 76MB file without significant detail reduction. Now I'm trying color reduction to 16 bit which should lower PNG file sizes little bit. This game is using true color space only for lighting effects. Overal image quality loss is negligible.
Maxim: Compression takes more memory than uncopressed image. 8416 x 12674 px image (200MB in BMP) takes 1,3GB. I've tried Ken's forum few days ago but posting there ended with 403 error.
-
That's right, I keep forgetting Microsoft doesn't want you to use the available memory in your system. At any rate, unless pngout is compiled with a special header, the 2 GB limit will stand on a 32-bit MS OS. I have access to 64-bit Linux machines, so I could try to help you out there once you have the final version of a map done. A bitmap compressed with 7-zip probably wouldn't be too bad of a size to send across the internet.
-
You can use all the available memory in your system, just not more than 2GB in user address space in the same process. They had to split it somewhere... and making it large address aware is more complicated than just a header setting.
Uncompressed images are generally stored at 32bpp for speed reasons, that's 4 bytes per pixel. 8416x12674 therefore takes 407MB. PNG compression at its simplest level would only need another 32KB of RAM for compression, but PNGOUT stores the compressed image in RAM too; so the rest of the RAM is presumably used for compression optimisation purposes. Given how easily it gets near the 2GB limit, he ought to release a 64-bit version anyway.
-
So I've done some research on Google Maps API and found user friendly way how to open big maps within Google Maps:
- GMap Image Cutter http://www.casa.ucl.ac.uk/richard/GoogleMapImageCutter/gmimgcut-download.asp (http://www.casa.ucl.ac.uk/richard/GoogleMapImageCutter/gmimgcut-download.asp) will create one HTML file and folder with JPG tiles
- Key for domain which will host final map is needed from Google website http://code.google.com/apis/maps/signup.html (http://code.google.com/apis/maps/signup.html) (keys are free)
- Key is inserted to HTML and everything is uploaded
Unfortunately there are some issues: Tiles are saved in JPEG so quality is not perfect and there is no 100% zoom ratio. For example zoom level 5 is 95% and zoom level 6 is 102%. I've tried to change picture size to multiples of 256 (tile size) but with no luck.
I've coverted that big map from AS2 and uploading it here: http://doc.riotix.com/map/ (http://doc.riotix.com/map/). Size of whole thing is 132MB and almost 22 000 files. Aside some problems I think this is a nice and easy way to store large maps. Even perfect if 100% zoom level would be created from PNG's...