Not signed in (Sign In)

Vanilla 1.1.4 is a product of Lussumo. More Information: Documentation, Community Support.

Welcome Guest!
Want to take part in these discussions? If you have an account, sign in now.
If you don't have an account, apply for one now.
  1.  
    I ran into a new issue yesterday. This may be a known problem, but it's one I've never encountered before (possibly because I never added a hex grid to a map).

    I created a map and exported it. Then yesterday I added a hex grid to the same map. Adding the hex grid (and changing the order of sheets a little) was the ONLY change I made to the map. I then exported it, using the EXACT same settings as before. It came out about one-third physically larger than the original in each dimension (by which I mean it went from a little under 7000 pixels wide to over 9000 pixels wide) and it also exported as a BMP rather than a PNG - meaning it was almost four times as large in terms of memory usage! I triple-checked, and I told it to export as a PNG.

    Does anyone have an explanation for this? I've never encountered this issue before - the maps always came out the exact number of pixels I entered into the export options, in the exact file format I entered. Is this a known quirk with hex grid maps? Is there something I need to do to compensate for it?

    ETA: I exported the non-hex version of the map several months ago. Could this maybe be a change caused by the latest update?
    •  
      CommentAuthorMonsen
    • CommentTimeJan 16th 2018
     
    Is the file named YourFilename.PNG.BMP ?
    Did you export with anti-aliasing on?

    If both those are true, what happened is that imagemagick failed to do the final conversion of the file. CC3+ always exports the file as a .bmp, and lets ImageMagick do the final conversion. Likewise, if antialiasing is on, the file is exported larger than the values specified (exact size dependant on antialiasing settings), and the image is then reduced to the correct size as part of the antialiasing process.
    However, if ImageMagick fails for some reason (usually memory issues), this proces doesn't take place, and you'll be left with the raw export from CC3+ in BMP format.
  2.  
    Yeah, it looks like you identified the exact problem. Thanks!

    When you say "memory issues" - would it likely be with my computer, or the program itself?
    •  
      CommentAuthorMonsen
    • CommentTimeJan 16th 2018
     
    It is a case of ImageMagick running out of memory when processing the bitmap. Like CC3+ itself, the version of imagemagick that ships with it is also 32-bit, so it is limited to 2GB (or 4 if it is LAA, not sure).

    This SHOULD be fixed with the latest CC3+ update, since it does pass some parameters to imagemagick to have it limit memory usage to avoid this exact situation, which seems to work for me at least.

    I do recommend you restart your computer before trying again, just to clean up memory and make sure as much as possible is available.
  3.  
    Just FYI, I tried exporting the non-hex version of the map, which I successfully exported back in May. (Before I did this, I reinstalled the latest update and restarted my computer.) The map came out borked exactly the same as the hexed version: too large and still a BMP. So either something has gone wrong with my computer since May, or one of the updates since then created a problem rather than fixing one!
    •  
      CommentAuthorMonsen
    • CommentTimeJan 16th 2018 edited
     
    What happens if you export the map at a significantly lower resolution? Same problem, or does it work then?

    And you are running CC3+ V.3.80 right?
  4.  
    I'll try it and see.

    Yes, it is CC3+ V.3.80.
  5.  
    Let me ask this: did CC3+ always default to 96 dpi, or was this increased in one of the recent updates? I never paid attention to it before. Maybe that's the problem.
    • CommentAuthorJimP
    • CommentTimeJan 16th 2018
     
    You can set your output. Once you 'Save As' a graphic type such as jpg, png, or bmp; look to the lower left of the Save As popup and open Options.
  6.  
    Right, but I'm trying to figure out what might have changed since May that is causing my maps to come out borked. Do you happen to remember if it always defaulted to 96 dpi?
  7.  
    I reduced the image dpi from 96 to 48 and got pretty much the same result as before. This is the error message being displayed at the end of the export box: "-limit memory 256 mb -limit map 512 mb -monitor -units PixelsPerInch (my file name) resize 7251x5436 -filter Bartlett -density 48 (my file name)"

    I would uninstall the last update, to see if that's the problem, except it doesn't show up on my "Programs and Files" list. To get rid of it, I would have to uninstall everything and reinstall (I still have updates 10 and 12). The prospect of doing that has got me pretty grouchy. This was a map that exported just fine a few months ago.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018 edited
     
    Posted By: Blackadder23I reduced the image dpi from 96 to 48 and got pretty much the same result as before.
    Sorry, I should have been more clear when I said lowering the resolution, as that can mean two different things, but I wanted you to try lowering the actual pixel size of the image, for example halving both height and width, thus exporting a smaller image, just to see if the behavior changes.



    Posted By: Blackadder23-limit memory 256 mb -limit map 512 mb -monitor -units PixelsPerInch (my file name) resize 7251x5436 -filter Bartlett -density 48 (my file name)
    Doesn't anything appear after this message? This is the command line being sendt to imagemagick, and after this line, it would typically report the progress on various tasks, like loading and resizing the image. Normally, the export should continue working on the process for some time after diplaying this line.


    Posted By: Blackadder23I would uninstall the last update, to see if that's the problem
    Updates can't be uninstalled. They have replaced vital files in your CC3+ directory, so uninstalling an update would remove these files and cause CC3+ to stop working because it would be missing it's files. Uninstalling something under Windows doesn't mean to put things back that was there before, just to remove what was put in.
  8.  
    Nothing appears after the lines I quoted. The exporting process just freezes, leaving me with a bit map file as I described before. I tried waiting quite a while, but it stayed frozen. Needless to say, this never happened before (I watched the exporting process more than once in the past, and I never even saw the message I'm quoting).

    I will try a smaller export.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018
     
    Posted By: Blackadder23and I never even saw the message I'm quoting
    The message is new with 3.80, it is just an informative message to see what is going on.


    Nothing appears after the lines I quoted. The exporting process just freezes
    Sounds like it may have problems starting imagemagick or something. If you look at my attached screenshot, you see how it is supposed to look during the export. Right now, I am wondering if either ImageMagick is still working in the background, just not displaying progress, or if it really hung. You said you tried waiting for quite a while, how long is that? Depending on a lot of factors, the final part of the process (after it shows the command line) can easily take up to 20 minutes on some systems (or more for a low hdd or very large export). I am wondering if it is perhaps working in the background, because normally when this process fails, it just exists immediately, the export window doesn't stay on screen. Exporting using a smaller resolution should partially answer that.
      Untitled.jpg
    • CommentAuthorJimP
    • CommentTimeJan 17th 2018
     
    I'm getting a similar error, anti-alias is off. I tried a map I worked on yesterday with no problem. Same error.

    It looks like even though I have turned off Defender, its still messing with things, and thats the problem. I have better anti-virus software.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018
     
    Posted By: JimPI'm getting a similar error, anti-alias is off.
    With simialr error, do you mean you experience the same as BlackAdder23, i.e. that exports don't complete properly, and you end up with a .bmp file (instead of a .png)?
    • CommentAuthorJimP
    • CommentTimeJan 17th 2018
     
    it doesn't complete. Then I get a popup from Defender that tells me my computer is protected. I dislike stupid software.

    It does leave a filename.png.bmp. 1200 pixels wide.

    I can convert it to png using Irfanview. Its only done this today.
    • CommentAuthorJimP
    • CommentTimeJan 17th 2018
     
    Odd. It suddenly started working.

    I did do a reboot, and it didn't work, then it started working correctly.

    I probably need a way to delete or make helpless Defender.
    • CommentAuthorJimP
    • CommentTimeJan 17th 2018
     
    I found a means to do so.

    Use at your own risk. These tenforums are run by people who are far more expert at Windows than I am.

    https://www.tenforums.com/tutorials/3569-turn-off-windows-defender-real-time-protection-windows-10-a.html
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018
     
    Out of curiosity, have any of you enabled the new 'Protected Folders' feature in Windows defender? This allows access based on a predefined program whitelist, so I am just wonder if there may be blocking access from the ImageMagick executable:
      Image1.png
    • CommentAuthorJimP
    • CommentTimeJan 17th 2018
     
    No, as I turned Defender off months ago. So I have no idea why it thinks it needs to do anything. New definitions were downloaded today, which is likely the problem.

    My trust of MS has dropped almost to zero.
  9.  
    Okay, I tried exporting it at half size, and initially it appeared to freeze on that same message exactly as before. However! I left it alone for a while and it apparently it finished when I wasn't looking. The result was a proper PNG of the designated size.

    Now I'm going to try to export a full size version again, and leave it overnight (if necessary) to see if it ever completes.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018
     
    Interesting. That indicates that it seems to be working fine, just that you don't see the status messages from ImageMagick for some reason. We'll still need to see the result of your full export before coming to any conclusion, but if that goes through, it seems to be just a display issue.
  10.  
    Maybe it's because I'm running Windows 7? I actually see a black box with white writing, rather than what you posted above.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 17th 2018
     
    Yea, mine is also normally black, that screenshot was for my work computer, where I have reversed the colors on the command prompt because it makes it easier to read on a projector in the classroom. The colors in that window follow the default command prompt colors in windows, so you can make it what you want.
    • CommentAuthortaustinoc
    • CommentTimeJan 17th 2018
     
    I'm wondering if the change is caused by the recent security update for the Spectre and Meltdown vulnerabilities. Older machines are reported to have a bigger slowdown for certain types of operations.
  11.  
    It took 45 minutes just for the last, "converting" step... but the map did eventually export at full size. It appears identical to the copy I exported in May.

    Now to try the hexed version (although I see no reason to believe it will be any different).
  12.  
    The hexed version exported as well, taking about the same amount of time. So the takeaway is to be prepared to wait some time for the final step of the exporting sequence. The time it took to export the non-hex version was almost double what it used to be. Hopefully there is some compensating advantage to the new setup.

    Anyway, thanks for your help.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 18th 2018
     
    The export times are longer than they used to be, true, but it also allowed CC3+ to export larger images than before. It is possible to improve the speed by replacing the version of imagemagick shipped with CC3+ with a 64-bit version instead, and then change the command line to remove the memory limits. You can also change the command line to change the filter used by by ImageMagick for the resizing (for antialiasing purposes), this will allow you to sacrifice quality for speed. All that is a little technical though, and requires manual editing of CC3+'s config file, so give me a word if you want to attempt that, and I'll post some instructions.
  13.  
    I appreciate the offer, but I definitely don't want to sacrifice quality. I just need to remember to start the export and leave it for a couple of hours. :)
    • CommentAuthormattekure
    • CommentTimeJan 18th 2018
     
    That sounds like something I'd like to do, could you post the instructions for it here?

    Posted By: MonsenThe export times are longer than they used to be, true, but it also allowed CC3+ to export larger images than before. It is possible to improve the speed by replacing the version of imagemagick shipped with CC3+ with a 64-bit version instead, and then change the command line to remove the memory limits. You can also change the command line to change the filter used by by ImageMagick for the resizing (for antialiasing purposes), this will allow you to sacrifice quality for speed. All that is a little technical though, and requires manual editing of CC3+'s config file, so give me a word if you want to attempt that, and I'll post some instructions.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018 edited
     
    Sure.

    - Close down all instances of CC3+
    - First, install a 64-bit version of ImageMagick on your computer. During installation, make sure to enable the checkbox to install legacy utilities.
    - Then, copy the entire contents of your imagemagick installation directory into c:\program files (x86)\ProFantasy\CC3Plus\ImageMagick (you may wish to take a backup of this directory first in case you wish to restore things without reinstalling CC3+)
    - Now, in your CC3+ data directory, edit the CC3Pluscfg.xml (remember this is an xml file, and it is very sensitive to mistakes. Don't make any)
    - Add the lines in red to this file. The lines in black are there already and are just there to help you position the lines in the file. This will remove the memory limits imposed on ImageMagick (and should only be done on a 64-bit version of it). Without memory limits, the export goes much faster.
    - You can also add the line in blue. This disables the Bartlett filter when processing the image, and sacrifices a bit of quality for a bit of speed. You can also use another filter of your choosing by typing -filter FILTERNAME in between the tags, but I don't have a list of these handy.


    <Software>
    <EvolutionComputing>
    <CampaignCartographer3Plus>
    <ImageMagickCmdLine>
    <memory></memory>
    <filter></filter>
    </ImageMagickCmdLine>

    <XP>
    • CommentAuthormattekure
    • CommentTimeJan 19th 2018
     
    Wow, that was a big speedup for me. 10000x10000 exported in about 15 seconds.

    Thanks!!!
    • CommentAuthorTonnichiwa
    • CommentTimeJan 19th 2018
     
    And how is the quality after that fast export?
    • CommentAuthormattekure
    • CommentTimeJan 19th 2018
     
    for me I couldn't tell any difference in quality. I didn't change the filter, just changed to the 64 bit version without the memory limit. I did some further testing and did a 20000x20000 in just under 2 minutes.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018
     
    Did you use the Q8 or Q16 version of ImageMagick? I've been using the recommended Q16 version myself, but I dug a bit into it today, and it seems like the Q8 version is a fair bit faster, and the colorspace used in CC3+ exports is only 8 bit per pixel component anyway, so that isn't a problem.
    • CommentAuthormattekure
    • CommentTimeJan 19th 2018
     
    I used the Q16 version.

    For the install, does it only use the convert.exe? I was wondering if the portable version download could work as well, just taking the convert.exe from that download.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018 edited
     
    Depends. If you grab a static build, you only need to copy this file, but for a dynamic build, you would need convert.exe and all the .dlls.
    I noticed that there wasn't a portable Q8 version, so I'll just probably going to grab the installer for the static version, install it, and then just extract convert.exe
    • CommentAuthormattekure
    • CommentTimeJan 19th 2018
     
    The I believe the portable one is static and doesn't touch the registry. I tried moving the convert.exe into an empty directory and it runs fine.
    • CommentAuthormattekure
    • CommentTimeJan 19th 2018 edited
     
    I just tried the Q8 static installer and it didnt give the option to produce the legacy utilities
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018
     
    How boring. I see the dynamic installer does, but that means you need to copy all the .dll's too. The final file size should still be about the same though, just a couple more files to handle. I haven't done testing myself though, so I don't know if it is really worth the bother or not.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018
     
    So, I got curious about the performance of the different options here, so I did a more thorough test. What I did was test four different ImageMagic configurations (64-bit Q8 version, 64-bit Q16 version, 32-bit Q16 version (the one that ships with CC3+) using default memory limits, and the 32-bit Q16-version without memory limits. All tests were repeated multiple times to get the most accurate results, and using precision measuring tools.
    The test I gave it was identical to what CC3+ has it do, that is, take a .bmp image, reduce it (to provide antialiasing) and convert it to .png, using the exact same command line CC3+ itself uses. Do note that this is only a test of the ImageMagic part of the export process, the part which happens after CC3+ is done doing the passes and rendering each sheet, the part that happens after the 'Converting to final format' message appears in the export dialog.
    The image I used was a 10908x11111 bmp that was converted to a 9817x10000 png. I did not touch the filter option, so the quality of all these are exactly the same (Do note however that the quality difference by using a difference filter is not huge, it will be barely visible, if visible at all, but for this test, I stuck with the high quality filter used by CC3+ by default)

    Now, when you look at the results below, you may wonder why the version used by CC3+ is so slow, when it could be so much faster. The answer to that is two important facts
    - CC3+ is 32-bit, so it ships with a 32-bit version of ImageMagick. This makes sense, because the shipped ImageMagick must be able to run on the same computers that CC3+ itself runs, can't have some customers not being able to export files.
    - Without memory limits, ImageMagick fails on some images, and this is dependent on the complexity of the image, and not purely the pixel size. While speed is nice, in the end, it is more important that the export is always successful. ImageMagick (normally) prints progress to the export dialog, so even if it is a bit slow, it is normally quite clear that it is chugging along.

    So, here are the results
    VersionAverage time (seconds)Comments
    Q8 x6420.6364-bit version. The Q8 version only handles 8 bits per color component in the image, but png, bmp and jpg all use this, so this is NOT an issue when used with exporting maps from CC3.
    Q16 x6429.2964-bit version
    Q16 x86337.4832-bit version. This is the version that ships with CC3+, but with memory limits turned off in the config. While turning off memory limits speeds up processing, it also means that every now and then CC3+ will produce an export that cannot be handled correctly, causing ImageMagick to run out of memory, and thus the export to fail.
    Q16 x86 (With memory limits)659.15This is the default configuration shipped by CC3+. The memory limits slows down processing, but ensures that ImageMagick never runs out of memory and fails the export
    • CommentAuthorTonnichiwa
    • CommentTimeJan 19th 2018
     
    This is interesting but I do have to say that when I tried to export a very large image a few times the export dialog did not move for two hours. Not so much as one number so I was unsure of what was going on. The next thing I knew it was done. The image size was 8000x8000. I was surprised to see that the export dialog was doing nothing but everything was still happening in the background.
  14.  
    I installed the Q16 X64 version, and it seemed to eliminate all the extra time that Image Magick was adding to my exports. Quality was unchanged (although I didn't mess with the filter). I'm satisfied with this resolution of the issue, so thanks again.

    One question: will I need to edit the CC3Pluscfg.xml again every time I run an update, or will the changes remain in place?
    •  
      CommentAuthorMonsen
    • CommentTimeJan 19th 2018
     
    Normally not. That file contains all your user configuration, so unless there is a particularly good reason, that file shouldn't be overwritten by an update.
  15.  
    Thanks.
    •  
      CommentAuthorMonsen
    • CommentTimeJan 20th 2018
     
    Posted By: TonnichiwaThis is interesting but I do have to say that when I tried to export a very large image a few times the export dialog did not move for two hours.
    I don't know why, but there seem to be situations where ImageMagick doesn't report it's progress to the export dialog. I've never known this process to hang though, so it is usually just a matter of waiting. If it crashes, that also terminates the export dialog immediately.
  16.  
    Even though ImageMagick is working much faster since I applied the fix suggested in the thread, it still isn't reporting any progress beyond the first message. Just FYI.
  17.  
    I did it. It took me +- 25min to render a 8000x6400 map with 50% antia. Just a question. It is to still apperas this mensagem in the end of the process or I Did something wrong?

    My Cfg
      Screenshot_14.jpg
      Screenshot_22.jpg
    • CommentAuthorjslayton
    • CommentTimeJul 16th 2018 edited
     
    That message is the command line passed to ImageMagick for debugging purposes. It is completely normal to see this message.