Is dropping indexes ever not safe?

One way that file sizes can increase is when fields are indexed, but don't need to be. By default, FileMaker fields are set to be indexed automtatically:

A common scenario I've encountered is that while developing, you do a search on a field. This creates an index for the field. Later, it turns out this index is never used again, but it remains in the file, bloating the file size and reducing the speed of new record creation.

Several FileMaker tools have the option to drop indexes and set them to be automatically rebuilt "Later (as needed"), as shown here in the Recover command:

My question: is this ever a dangerous option?

I'm imagining a case where a field index was necessary due to some sort of complicated relationship (perhaps a Layout in file A that is from a table in File B), or perhaps a relationship using ESS - could FileMaker fail to realize the index needs to be rebuilt?

Anyone seen this happen? Or, will FileMaker always rebuild every index when it's needed, without exception?