Perceiving causality, anticipating events, choosing an action or understanding language - all facets of complex behavior are anchored in time and rely on representing, recognizing and expressing temporally patterned sequences. Established approaches in the cognitive sciences formulate these processes in terms that involve the manipulation of sequentially organized time-discrete (symbolic) representations. While we understand fairly well how artificial systems perform on different aspects of the problem, existing models of biological (specifically spiking) networks that use biophysically realistic learning rules typically focus on individual task features (e.g., acquisition of serial order, temporal rescaling, etc.) and often employ architectures and biophysical properties that support specific functionalities. This makes it very challenging to compare them and identify their weaknesses and how to improve them. To bridge this gap, we present a conceptual and computational framework for benchmarking and comparing biologically-constrained models of sequence processing. Leveraging psycholinguistic paradigms such as artificial grammar learning (AGL), we propose tasks to assess diverse capabilities expected from a general sequence processor. The symbolic formulation provides a methodology for generating sequences of controlled complexity along with grounding in formal analysis. This allows us to probe a model's ability to process complex inputs systematically, quantitatively, and in a system-agnostic manner. As a proof-of-concept, we use the framework to compare several prominent models proposed in the last decade. Testing these models beyond their original scope and systematically scrutinizing their behavior and task performance in a series of controlled experiments reveals their strengths and limitations, demonstrating that they primarily perform well at the tasks they were designed to solve but typically fail to generalize beyond their intended purpose. None of the tested models showed convincing performance across multiple features not included in the original studies. As such, they do not provide a sufficiently detailed understanding and integrated account of the biophysical mechanisms underlying sequential processing. Through this meta-analysis, we aim to provide a critical evaluation of current models and synthesize their insights into a set of functional and neurobiological features to be corroborated with experimental data and guide future studies.