SSL Issue: Can't Connect to FQDN (LAN), and Name Match Issue

I'm having an SSL issue that I've not experienced before and hoping the community can help. I've searched and found LOTS of posts across the FM communities, but so far I've not found the fix.

FileMaker Server 18.04.xx on Mac OS X server (Mojave)
Clients: FMPA 18 and 19 (Windows and Mac)
GoDaddy SSL
LAN access only

We have an SSL certificate from GoDaddy for a FQDN (fms.ourdomain.com) and the installation appeared to be "successful". This is the first new/custom SSL on this particular server. I installed the two .cert files, one containing "bundle" in the file name. I did nothing with the .pem file. Was I supposed to append the .pem contents to one of the others? I've not used GoDaddy SSLs for FMS in a very long time.

Naming the server to "fms.ourdomain.com", closing all DB files, restarting the server (several times) and eventually restarting the Mac:

  • I cannot connect to the host by typing the FQDN into the Favorites dialog (including after deleting all favorites from the list) "Connection Failed"

  • And of course I cannot connect to the Bonjour generated option

  • ConnectionStatus is "2", orange lock when I connect via IP address

  • When connected by IP address (and orange lock) the application works, except for PDF files (interactive) in containers (mission critical) and .mov files won't play (not real critical now).

"( Proxy

GET

/Proxy/AB2FC9863B6BBE4FE5414CEC4E5FF0AC.pdf?urlid=AB2FC9863B6BBE4FE5414CEC4E5FF0AC

1631

-H "Upgrade-Insecure-Requests: 1"", etc., etc.

  • If I change the server name back to the pre-production name "Bla-Bla-MacPro" (not exactly) I can connect via Favorite or Bonjour local, but of course the name doesn't match the SSL, so status "2". However, PDFs in containers are still broken.

Any guidance would be greatly appreciated. I've wasted a lot of time on this and it's interrupting production. I'd like to get it worked out to status "3" (green lock), but that's less critical than the PDFs and reliable connectivity.

Could you post the content of the 'Access' and 'Event' log for the period when a client connection is being established and a container and other fields are being accessed? We should get some hints from the log files.

Hi, Torsten, thank you for your reply.

I did look at the logs and did not catch anything, but perhaps you and others will see something I missed. In fact I tried a couple things since looking at the logs, so perhaps something new will show.

I’m away from my office now (7:15 PM here) but I’ll post them ASAP.

I would like to know some things in more detail:

  • LAN access only means that the FMS is only accessed from the intranet?
  • Is the domain name in the internal network "ourdomain.com" from your example?
  • Did you enter the FQDN in the admin console?
  • How did you install the certificate? Admin Console, Command Line or by placing it in the CStore folder?
  • Is the FQDN of the server entered in the internal DNS server?
  • normally you need 3 files for installation: certificate, bundle and a key file – you did not mention a key file

There can be several reasons for your issue. The answers to these questions can help to narrow down the problem.

mipiano, thank you for your reply.

Installed via console, and “yes” to everything else, except this location does not have a DNS server, and the router does not allow DNS config specifics as needed (common issue).

Key file was installed as well.

This is where the problem will come from: without a DNS server, the individual client computer does not know what "house number" (IP address) the server has. A DNS must do the name resolution and report the IP address back to the client. Otherwise, no connection can be established.

And if you use the IP address directly, you can connect, but without using the certificate. That's why you don't get a green lock then.


Edit: If you have another Mac that could be always on, you could use it as a name server (DNS). Here you find a nice little software for this purpose: https://cutedgesystems.com/
The app is called DNS Enabler. It’s relatively easy to configure. Send me a PM if you need help.

Best
Udo

3 Likes

Thank you. Yes I know this. The green lock is less important than being able to upload and view PDFs and such. (Related, I know)

Many companies do not have dedicated internal DNS servers.

I will likely revert to the test certificate or a self signed until a solution is defined. The client is not prepared to establish a dedicated DNS server as it’s a small startup.

I have never experienced this mix of conditions; i.e. can’t connect via FQDN, etc.

The DNS could also run in a virtual machine. You don't necessarily need extra hardware for it. I haven't tried it yet. Maybe the service could be installed parallel to the FMS. The problem should be easy to solve!

Replying to your edit:

Yes, I am already considering this as a temporary solution. It’s quite fragile as it injects another dependency. At least an option.

Don’t worry – this software is stable. I use it myself.

When the new Mac mini comes that I have ordered today, I will use it as a FMS and try to install DNS Enabler parallel to the FMS. I will get back and report. I think my new Mac comes in the next 3–4 days.

1 Like

I manage many FMS servers. I’m not concerned about the software “working”, I’m only looking for a fix for this one issue. Thank you.

Please excuse me, I didn't mean to offend you.
I just wanted to say that I don't think this solution is fragile, as you said.

My English is not very good. It can happen that I put my foot in my mouth.

How do things work when you use the server's IP address instead of the FQDN?

No worries at all, mipiano (Udo). I’m not offended. I appreciate the input. :slightly_smiling_face:

Torsten, the application functions normally except for PDFs in interactive containers (important) and mp4 files aren’t viewable. There’s an SSL warning and Orange lock (as expected).

Interestingly, I rebooted the Mac last night and now the lock is still orange, but clicking it reveals that the SSL cert is "trusted for 192.xxx.xxx.x" where it previously reported a mismatched name. But, the PDF files are still not accessible (to insert or view).

Update: We have functional access to the PDF documents (sort of) after making some changes to the Mac OS firewall "allow" list and removing the interactive declaration. Interactive PDF is not available (visually blank container) since adding the custom SSL certificate, and for at least some we need to be able to scroll multi-page PDF files.

The SSL still shows "trusted for 192.xxx.xxx.xxx" rather than "mismatched name". We're connecting by IP address until we fix the internal DNS issue. I'll decide how best to provide DNS server support to address that.

Dale,
as others have pointed out you have NO problem with Certificates, but you have one with DNS.
Have you on a local machine tried to use the command Traceroute in terminal.. traceroute yourserver.com.
What does it say? Probably an error.
Have you on your Router included a port forwarding, for the server ports 80,443, 5003,16000 to your local IP?
If this is done.. the machines would go out to find yourserver.com.. hit the local router at 192.0.0.1 say.. this guy does not know anything where the machine is.. so it goes out to where your Nameserver is.. say at AWS or you ISP..there it says.. go back to the static IP of youserver.com.. and there it hits the port forwarding rules you have decided.. and goes to the right machine..

Hi, Pierre, thanks for your input. Yes, there is a DNS problem as there is no DNS server, nor external (WAN) access to this particular server.

Since the original post several things have been adjusted and the symptoms have minimized. When I first posted there were a few things happening (all since the new SSL cert), the most problematic being the PDF file problem. That was partly due to Mac OS firewall settings (but wasn't a problem with the default SSL). A past IT person had not allowed all of the FM processes. That's fixed now, but interactive PDFs and MP4s are not available. I've yet to resolve that.

Some port forwarding will cause other conflicts in this particular setting, so a DNS server (possibly Raspberry Pi) will be the likely route we'll take. (I've found relying on routers for this to be a bit fragile, but if the assigned router had NAT Loopback (hairpinning) I'd use it for a while .) Also, I don't want traffic to go out to WAN and back, so that brings me back to local DNS server as well.

Thank you to all, for your input. The process helped me to track down a couple of quirks. I still need functioning interactive PDF containers (was working with the default FM SSL) and I'm not yet sure how to resolve that. I know it's an issue for many users.

1 Like

Hey Dale,
that your PDFs not work IS a direct problem, that your clients do not find the server.
An Internal DNS is one option.
Allow external access to yourdomain.com with port forwarding, your other, more simple option, does not slow down the access.. I only will resolve all your issues. The TCP IP Protocoll will take the best route... once it found the target. Opening the ports on your router avoids having, maintaining DNS servers.. And IMHO (haha.. I love that one.. "humble"..obviously people who use that think they are speak out of the burning bush ).. since you have already a mac on the net.. I would with the cuteedgesystems approach, unless you are savvy in Rasperry PIe or willing to dig in.

1 Like