I hear this question quite a bit: "We have over one hundred people with View All Data which is too many. How can I reduce this number to reduce the risk that people can view more than they should be able to?"
Before you can reduce the list of who has View All Data, you may need to determine if any of those users have this permission for one of the following ten reasons listed below. If they don't need to accomplish one of these tasks, it's a good idea to change their profile or permission sets over to something that doesn't have View All Data.
You need Read on all objects (Sharing is ignored in this case) - this is the primary use case for granting View All Data and what View All Records was meant to solve for by ignoring sharing on an object basis rather than across all objects.
You need to set the Running User of a dashboard to someone other than yourself (requires Manage Dashboards permission as well)
Anyone with Modify All Data, Author Apex, Download AppExchange Packages or Create AppExchange Packages will also get View All Data (it's required).
The "big deal alert" user on opportunities needs to have view all data
You need to view private attachments, notes, and Google docs from other users will be displayed in the related list and in the API
You need to view private events and tasks owned by other people are available in the calendar and API.
You need API Access to Forecasting
You need to do an API Upsert on a non unique fields
You need to use the System (Apex) system log (in the header)