The availability of high-density electrophysiology is transforming systems neuroscience and enables virtually any experimental lab to simultaneously acquire activity from hundreds of neurons throughout the brain. However, extracting single-cell activity from the resulting large datasets remains a major challenge: different experimental settings introduce various sources of contaminating noise, and even optimized spike-sorting algorithms still require manual evaluation of the resulting spike clusters. Establishing a pre-processing pipeline and manual evaluation requires expert knowledge and is often extremely labor-intensive, presenting a major roadblock, particularly for experimental labs with little programming experience. To evaluate and improve the performance of automated preprocessing pipelines, we, therefore, used Neuropixels probes to collect various datasets, including high- and low-quality recordings from multiple setups, and combined recordings with optogenetic stimulation or functional imaging. We then tested different preprocessing methods and hand-labeled the resulting spiking clusters to characterize the impact of preprocessing on cluster quality. Different analyses, such as rescaled median subtraction and automated channel rejection, significantly reduced the number of noise clusters but, importantly, even high-quality datasets still contained a large amount of non-neural noise clusters. We, therefore, established a set of noise-predictive quality metrics, such as hyper-synchronous spiking across clusters, and used supervised UMAP analysis and an SVM classifier to automatically identify noise clusters. The classifier detected hand-labeled noise clusters in unseen datasets significantly better than existing methods (84% versus 62% cross-validated accuracy) and generalized well across all dataset types. A similar approach isolated single-unit clusters and we created a fast and simple interface for efficient evaluation of cluster quality. We demonstrate the importance of evaluating high-density spike sorting outputs and offer an automated processing pipeline, including novel methods for data denoising and classification. The pipeline is very user-friendly and extendable with additional metrics, thus providing a powerful tool to efficiently isolate single-cell activity from large datasets.