Have you changed the effects? When I start a new DD3 map and draw a cave using the cave tools, I get the result as you can see in the image below. The glow serves to make the illusion that the cave is cut into the background.

From your image, it looks more like you have an edge fade effect on the FLOORS sheet instead of a glow effect.

Note that if you do want walls for your caves instead of this effect, and if it is for more than a on-off occasion, remember that you can edit the cave drawing tools by clicking Advanced in the drawing tools dialog. From here, you can select a tool, then hit the new button to clone it, and an then you can use the outline option for your new copy to have the tool draw a wall for you as well as a floor, just set the outline option to draw an extra entity, and set it up with the properties you want. (SHEET and LAYER should be WALLS)

One potential problem could be windows UAC causing this. UAC often causes a problem with network drives and installers, because even if the drive is mapped in your normal user session, it is not brought along to the elevated session windows uses when running installers.

That means that either the navlines or the landmass is on the wrong sheet, or your sheet order is wrong. Check the sheets of the entities with the Info menu -> List command, and make sure the sheet for the navlines appear higher up in the sheet list than the sheet for the landmass.

Not sure why they are not included anymore, but you can copy them out from the editing tutorial drawing @Tutorials\UserManual\Trims.FCW

FT3 saves these settings in its program directory, so the only way to change them if you have installed FT3 in the default directory is to run it with administrative privileges. You can do this by right clicking the program icon and choosing run as administrator. This only have to be done when changing these settings.