Crash due to out of memory

Running a server with 21.0.2.202 through FMPHost.com. They spin up virtual machines on AWS, and it's been great for a while. This week it's crashed 3 times, and I've had to call them to restart it. The first two times the guy just said it was just an AWS fault, but the third time I got a different tech and he looked closely and said it was a memory issue, so he suggested turning up the cache.

The Virtual Machine has 16GB RAM, and cache was set to 512, so he bumped it to 1024.

I looked at the Stats.log and it's usually pinned at 100%, with some dips to 95 or 97 (and obviously less right after restart).

Is that "bad" that I turned up the cache when it was already at 100%? I looked at event log and don't see anything happening around the time it crashed worth noting.

If you are hitting the cache most of the time - that's good! The cache is in RAM and it means that you don't need to search the disk looking for your data.

When your cache % is low you should increase your RAM, not the other way around.

A full cache doesn’t cause server crashes, but an out-of-memory situation definitely can. That said, 16 GB is usually more than enough. I suspect there’s a different issue at play here!

Jan

Do you have some crash reports?

Or monitoring?

Like did the machine run out of memory?
Which would also mean that swap file was full.

But what does crash mean?
e.g. if you run a script that allocates lots of memory on the server and the system kills the process eventually, it would just be restarted and continue.

Ok this is helpful. Cache didn't make sense to me, but this is the company who I pay to host so I just sort of assumed they knew of an issue I wasn't aware of. I have the Filemaker Logs but they don't show anything helpful. I've requested the server logs from AWS so I can see what the actual error is. I'm honestly unclear if it's a "crash" or a "hang" anyway. Just that it became unresponsive. Here's the logs I have, please let me know if I should turn anything else on.

a711783_AWS.log.zip (16.7 KB)
I got the AWS log from my host (attached), but I don't know what it means... Can someone help me know what to look for for a clue about what's causing the crash. Apparently, they looked at this and saw it was a memory error. They also said they could send me the system level log from Linux (I thought I was on Windows, but I guess the moved me at some point). Thanks in advance.

As far as I can tell, the VM did not crash but was properly rebooted. This doesn’t appear to be a FileMaker issue.

Just an aside comment for anyone interested: Some time ago @WimDecorte (and others) suggested activating Top Call Statistics on all deployments for various kinds of troubleshooting. We've done this to all of our servers and detect no performance issues (although our deployments are on premises servers, aside from certain test environments).

2 Likes