More info:
- it turns out the client was sending mass emails to several thousand customers with a link to the WebDirect site. I didn't know they were doing this, but once they told me this, the pattern became clear. Each email blast is followed by the proxy server crashing within in a few minutes. Logs show something linke 1500 hits to the
/fmi/webd/url within under a minute. We've repeated this several times now and it's reproducible. - It's not actually humans clicking the URL in the email to the WebDirect site, but rather some sort of email malware scanners accessing the URL.
- I don't know whether it's the volume of hits that is crashing the proxy or some other issue (such as a malformed HTTP request, bad headers, etc.) but in any case I'm working with the client to say "don't do that" (don't include WebDirect URLs in the emails).
- one bit of good news. I discovered that I no longer have to reboot the entire server, rather I can issue these commands:
fmsadmin restart SERVER
fmsadmin restart WPE
and the WebDirect comes back up. Still annoying as we have kick offline any FMPro users, but quicker & less disruptive then a full server reboot.