TypeSeg

See which content types are inside a text file, and where. TypeSeg labels regions of code, markup, data, and encodings across 35 textual content types. It shows where each type starts and ends, including in mixed or malformed text.

TypeSeg uses deep learning sequence modeling to predict a content type for each character. Its lightweight U-Net and Mamba models have 1.52 million and 1.36 million parameters, respectively, and run on local hardware. U-Net processes short overlapping sections; Mamba uses context from the whole file. About the models.

No GPU required. In our research-runtime benchmarks, U-Net processed about 292,000 characters per second on an 8-core AMD Ryzen AI 7 350 laptop CPU. With a consumer-grade RTX 5070 laptop GPU, we measured 4.8 million characters per second—about 4.8 MB/s for ASCII text. Inference runs locally. Benchmark setup.

A file detector such as Magika can identify textual files before TypeSeg labels their internal regions.

One file. Several content types.Illustrative ASCII file · 491 bytes

PythonPowerShellPythonBase64Python

0245491 bytes

Hover to unfold the file. Select a region to inspect its text.

Inspect source

Inside the fileSame bytes, unfolded into lines

1import base642from pathlib import Path3 4# A text file can carry several languages.5sample_name = "inspection-demo"6 7powershell_source = """8$files = Get-ChildItem -Path . -File9$files | Select-Object Name, Length10Write-Output "Inspection complete"11"""12 13# The encoded text below is a harmless example.14encoded_note = "VGhpcyBpcyBhIGhhcm1sZXNzIGV4YW1wbGUgZm9yIGNvbnRlbnQtdHlwZSBzZWdtZW50YXRpb24u"15 16decoded_note = base64.b64decode(encoded_note).decode("utf-8")17print(sample_name, decoded_note)18 

PythonPowerShellBase64

Hover across the bar to follow the highlighted text. Byte ranges match this source exactly; labels are illustrative.

Select a region to inspect its source. Use cases include text inspection, content-specific analysis, and dataset review.

Try the web demoInstall from PyPIStart with PythonGet the code

In your terminal

Inspect mixed text directly from the command line:

pip install typeseg
typeseg --model fast file.txt

Recorded Mamba output from TypeSeg on PyPI. The historical screenshot’s banner reads textseg.precise; the current API call is typeseg.precise. Colors mark content types; the table shows ranges and confidence. Shell command text is redacted.

The CLI guide covers models, piped input, and output formats.

Highlights

• Mixed and malformed text. Labels content types even when tags are missing or syntax is incomplete.
• 35 content types. U-Net achieved 94.9% F1 for character labels on 875 human-labeled files covering all 35 types. How the score is measured.
• Small models, local inference. U-Net has 1.52M parameters and runs on CPU or GPU.
• Character-level locations. Each segment includes a content type, start and end offsets, and confidence. Output format.
• Python, CLI, and browser. Try the Python package, command-line tool, or browser demo. Demo text stays in your browser.
• Open source. Code and general checkpoints are available under Apache 2.0. Research and citation.

Rates are warmed research-runtime measurements on an AMD Ryzen AI 7 350 and an RTX 5070 laptop GPU (85 W). Measurement details.