Not all value lists are created equal. While it's true all primary fields in a value list need to be indexed (and in most cases the secondary field) - there is a fundamental difference between an "all values" list, and a "show related values from" list.
The former is fast - if you have a value list showing all values from a field in a table - then the index is what is used for display. However once you choose related values, it is a query that is user-specific and the records used in the value list have to be downloaded from server to client - it's like any other operation such as sorting records, aggregating, or listing.
Yes value lists are implicitly global in most cases - with exception of conditional value lists which are context dependent. Value lists that display related values cannot be used in find mode whereas "all values" ones can.
One use I have utilised value lists for is to obtain sub-sets of primary keys from a table. Let's say I have a table with 100,000 records, and I wish to obtain a list of primary keys where the "type" field is of value "contact".
I could create a relationship based on the type field to relate to just contact records, and build a value list off that - but because it is related values, a query is performed and the resulting records have to be downloaded from server to client - the more records the slower the value list will take to display.
An alternative approach may be to create a stored and indexed calculation field on the table. This calculation would be:
if ( type = "contact" ; primaryKey ; "" )
The primary key of any given record is only indexed if it's type is contact. With this done you can now build an "all values" value list based on this calculation. Because it is all values, only the index of this field is sent from server to client (no record data) and as a result you have a list of primary keys without any record transfer. As far as I'm aware this is the only way I have found to actually transfer legitimate field information from a record without being sent the entire record.
There are obvious caveats to this method. One being you need to store and index the primary key for each scenario. If you want 20 different lists, that's 20 calcs - it will increase the overall size of the table, so like I say it's to only be used sparingly where the query is simple (and not dependent on any related tables so can be indexed) and where the resulting number of IDs is large (where record transfer is not desirable).