How Spotlight's Index Works on Mac (and How to Rebuild It)

You hit Cmd+Space, type the name of a file you saved yesterday, and Spotlight returns nothing. Or it returns the file but as a “web search suggestion” instead of an actual result. Or — most maddening — it works for some files but consistently fails on a specific folder. The index is broken, in some specific way that requires understanding what the index actually is.

Spotlight’s index is one of the more impressive engineering feats on macOS. It tracks metadata for every file on your drives, stays current as files change, and answers complex queries in milliseconds. When it works, it’s the fastest file-finding system on any consumer OS. When it doesn’t, it can be silently incomplete in ways that take forever to notice.

What Spotlight’s Index Is

Spotlight maintains a per-volume database of metadata about every file the system thinks you should be able to find. The metadata includes:

• File name and path
• Creation, modification, and access timestamps
• Type (UTI — uniform type identifier)
• Size
• Content — text extracted from documents, code, PDFs, emails
• Per-type metadata — image dimensions and EXIF, audio bit rate, document author, etc.
• Tags and Finder labels
• Comments (the “Spotlight Comments” field in Get Info)

The database lives at /.Spotlight-V100/ on each volume — hidden in Finder but accessible from Terminal:

ls /.Spotlight-V100/

You’ll see Store-V2 directories holding the actual indexed data. The total size depends on how much content has been indexed; on a heavily-used 500 GB drive, the index can be 5–15 GB.

The mds Process Family

Spotlight is implemented by a few processes that work together:

• mds — the metadata server, the master process
• mds_stores — handles the actual database read/write
• mdworker — child workers that scan and extract metadata from files

You’ll see all three in Activity Monitor. They fluctuate based on indexing activity. After a major OS install or a big file change, they can pin a CPU core at 100% for hours while reindexing.

Each file the system thinks should be indexed gets passed to a relevant importer plugin — a binary at /System/Library/Spotlight/ or /Library/Spotlight/ that knows how to extract metadata from a specific file type. There’s an importer for PDFs (extracts text), one for plain text, one for Microsoft Office docs, one for images (EXIF data), and so on.

Third-party apps can ship their own Spotlight importers. Adobe Photoshop ships one for PSD files. Logic Pro ships one for project files. When an importer is broken or missing, files of that type don’t get indexed properly.

How It Stays Current

Spotlight uses a kernel-level mechanism called fsevents to know when files change. Every modification — create, delete, rename, content change — generates an fsevent. The Spotlight daemons subscribe to these events and update the index accordingly.

This is why Spotlight is usually current within seconds of a file change. It’s not polling; it’s reacting to events.

Checking Index Status

The mdutil command tells you the state of indexing for any volume:

mdutil -s /

Possible outputs:

• “Indexing enabled.” — index is on and current
• “Indexing enabled. Estimated reindexing time: X” — currently reindexing
• “Indexing disabled.” — index is turned off (manually or by error)
• “Indexing and searching disabled.” — fully off, no metadata or search

For all volumes:

mdutil -s -a

This is the first thing to check when Spotlight isn’t working. If indexing is disabled, search will return nothing because there’s nothing to query.

Toggling Indexing

To turn indexing on for the boot volume:

sudo mdutil -i on /

To turn it off:

sudo mdutil -i off /

To turn it on for an external drive:

sudo mdutil -i on /Volumes/External

You’d turn indexing off for performance-sensitive drives where you don’t need search (e.g., a dedicated Time Machine destination, a video-editing scratch drive that’s wasted spinning up the index for cache files).

Power users use SweepSweep handles all the cleanup that articles like this describe. Get Sweep free →

Rebuilding the Index

When the index is corrupt (search misses files that should match, or returns stale results), the fix is to rebuild it. The right way:

sudo mdutil -E /

This erases the existing index for the volume and triggers a full reindex. It can take anywhere from 30 minutes to many hours depending on:

• How much data is on the volume (more files = longer)
• The mix of file types (PDFs and large databases are slow; plain code is fast)
• Whether your Mac is on battery or plugged in (mds throttles on battery)
• How much CPU is available

For all volumes:

sudo mdutil -E -a

During the rebuild, Spotlight searches will be incomplete or empty until the relevant data has been re-indexed. Plan for several hours of degraded search.

What mdfind Does

Spotlight has a CLI counterpart: mdfind. It queries the same index from the command line.

mdfind "report"

Returns every file Spotlight thinks contains or is named “report” on indexed volumes.

You can use the structured query language:

mdfind 'kMDItemContentType == "com.adobe.pdf"'

Returns every PDF.

mdfind 'kMDItemFSCreationDate > $time.today(-7)'

Returns files created in the last 7 days.

mdfind 'kMDItemAuthors == "John Smith"'

Returns documents authored by John Smith (where authorship is in metadata).

mdfind is faster than find for content-aware searches because it queries the index instead of walking the filesystem. For pure path/name searches, find is comparable.

Excluding Folders from the Index

System Settings → Spotlight → Search Privacy lets you add folders that won’t be indexed.

Common things to exclude:

• Code repositories with millions of small files (node_modules directories add up fast)
• Backup destinations with redundant copies of everything
• Old drives or shares full of files you don’t need to search
• VM disk images and their internal structure

The exclusions are stored in ~/Library/Preferences/com.apple.spotlight.plist and /.Spotlight-V100/VolumeConfiguration.plist. The latter is per-volume.

You can also exclude paths from the command line:

sudo mdutil -E /

(Wait, that’s reindex. There’s no clean CLI for adding privacy exclusions; the GUI is the cleanest path.)

Excluding by File Type

Spotlight can be told not to index certain content types. You’d do this if a particular file type is causing problems (e.g., an importer crashing, or you don’t want a class of file searchable).

The settings file is at /System/Library/Preferences/com.apple.SpotlightServer.plist, but Apple doesn’t expose a clean way to manage type exclusions through the UI. Most users won’t touch this.

Reasons Spotlight Misses Files

If a file isn’t appearing in search:

1. The volume isn’t indexed — mdutil -s /Volumes/Foo to check
2. The folder is in Privacy exclusions — System Settings → Spotlight → Search Privacy
3. The file was created during reindex — wait for index to complete and try again
4. The file’s importer is missing or crashed — for unusual file types
5. The file is in a hidden directory — Spotlight doesn’t index hidden folders by default
6. The file’s mdimporter ran but extracted nothing — Spotlight finds the file but has no content text to match against
7. The volume is noindex flagged — set by certain apps; check with getxattr

For a manually-flagged folder check:

ls -lO@ /path/to/folder

If you see noindex in the output, that flag is suppressing indexing. Remove it with:

chflags -R noindex,nohidden /path/to/folder

When mds Pins Your CPU

A major macOS upgrade triggers a full reindex. So does adding a large external drive, restoring from a backup, or using mdutil -E. During these, mds_stores and mdworker can run hard for a long time.

If it’s been more than a few hours and CPU is still pinned:

• Check Activity Monitor for which process specifically — mds_stores doing genuine work, or stuck?
• Check disk activity — high read/write means progress
• sudo killall mds mds_stores mdworker to restart the indexing daemons (they’ll relaunch via launchd)

The full restart is sometimes the right move when an importer is in a crash loop. It’s mostly safe — at worst you lose a few minutes of indexing progress.

There’s a faster waySweep does this kind of cleanup automatically. Try Sweep free →

Inspecting What’s Indexed for a File

To see all the metadata Spotlight has captured for a specific file:

mdls /path/to/file

Output is dozens of fields with values. Useful for:

• Verifying that content was extracted (kMDItemTextContent for documents)
• Checking why a file isn’t matching expected queries
• Debugging custom or third-party importers

If mdls returns mostly empty values for a file that should have content, the importer for that type isn’t working — either missing or crashed during indexing.

What Spotlight Doesn’t Index

By design, some things are excluded:

• Files in directories with the noindex flag
• Hidden files and folders (by default; can be overridden)
• The /private directory tree
• Time Machine backups (excluded automatically when on Time Machine destination)
• iCloud Drive files that aren’t downloaded locally (cloud-only)
• Files inside packages and bundles (Spotlight indexes the package as a whole, not its internals)

The package limitation is sometimes frustrating: searching for a file inside a .app bundle won’t return matches, even if you know it’s there. Use Terminal find for those cases.

Why It Matters

Spotlight is the file-finding mechanism on every Mac. When it’s working, you don’t think about it. When it’s broken, you waste minutes per search and still don’t always find what you need.

The simple maintenance flow:

• Check mdutil -s if search seems off
• Rebuild with mdutil -E / when the index is genuinely corrupt
• Use mdls to debug specific files that aren’t matching
• Watch for mds_stores in Activity Monitor when something’s running unexpectedly hot
• Keep Privacy exclusions current — exclude node_modules and big VM images so Spotlight isn’t burning effort on data you’ll never search

It’s a few-times-a-year tool to actively manage. The rest of the time, knowing how it works just makes you better at understanding what it returns.