Text Processing Tips for Developers: Count, Convert, Sort

Text manipulation is one of the most common tasks in development. Whether you’re counting words in user input, converting naming conventions, sorting log entries, or cleaning data files — having the right techniques at hand saves hours. This guide covers practical patterns for everyday text processing.

Counting: Words, Characters, Lines

Word Counting

The simplest word count splits on whitespace:

function countWords(text) {
  if (!text.trim()) return 0;
  return text.trim().split(/\s+/).length;
}

This works for most European languages where words are separated by spaces. For CJK text (Chinese, Japanese, Korean), word segmentation is a harder problem that typically requires a library like Intl.Segmenter.

In Python:

word_count = len(text.split())  # split() handles any whitespace

In the terminal:

wc -w < filename    # word count
wc -m < filename    # character count
wc -l < filename    # line count

Tip: wc is the Unix Swiss Army knife for text stats. Use wc -lwm to get lines, words, and characters in one call.

Try our Word Counter, Character Counter, and Line Counter for quick analysis without leaving the browser.

Character Counting Gotchas

Character counting seems trivial until Unicode enters the picture:

"Hello".length          // 5 — expected
"café".length           // 4 — correct
"👋".length             // 2 — surprise! (surrogate pair)
"👨‍👩‍👧".length            // 8 — ZWJ sequence

JavaScript’s .length returns the number of UTF-16 code units, not visible characters. For a user-perceived character count (grapheme clusters), use:

function countGraphemes(text) {
  const segmenter = new Intl.Segmenter("en", { granularity: "grapheme" });
  return [...segmenter.segment(text)].length;
}

Rule of thumb: use .length for storage/byte calculations and Intl.Segmenter for display/UI character limits.

Case Conversion

Common Developer Cases

Converting Between Cases in JavaScript

The key insight: first split into words, then reassemble.

function toWords(input) {
  return input
    .replace(/([a-z])([A-Z])/g, "$1 $2")  // split camelCase
    .replace(/[_\-]+/g, " ")               // split snake/kebab
    .split(/\s+/)
    .filter(Boolean);
}

const toSnakeCase = (s) => toWords(s).map(w => w.toLowerCase()).join("_");
const toCamelCase = (s) => toWords(s).map((w, i) =>
  i === 0 ? w.toLowerCase() : w[0].toUpperCase() + w.slice(1).toLowerCase()
).join("");
const toKebabCase = (s) => toWords(s).map(w => w.toLowerCase()).join("-");

Python: use the inflection library for robust conversions, or re.sub(r'(?<=[a-z])(?=[A-Z])', '_', s).lower() for a quick camelCase-to-snake_case.

Convert text between 9 different case styles with our Case Converter.

Sorting Lines

In the Terminal

sort filename              # alphabetical
sort -r filename           # reverse
sort -n filename           # numeric
sort -u filename           # unique (dedup + sort)
sort -t',' -k2 filename   # sort by 2nd CSV column
sort -V filename           # version/natural sort (file1, file2, file10)

In JavaScript

const lines = text.split("\n");

lines.sort();                              // alphabetical (ASCII order)
lines.sort((a, b) => a.localeCompare(b));  // locale-aware
lines.sort((a, b) =>                       // natural sort
  a.localeCompare(b, undefined, { numeric: true })
);

Watch out: JavaScript’s default .sort() uses lexicographic (ASCII) order where "10" < "9" because "1" < "9". Always use localeCompare with { numeric: true } for human-friendly sorting.

Removing Duplicates While Preserving Order

const unique = [...new Set(lines)];

unique = list(dict.fromkeys(lines))  # preserves insertion order (Python 3.7+)

awk '!seen[$0]++' filename   # preserves order, unlike sort -u

Sort, deduplicate, and shuffle lines with our Line Sorter.

Reversing Strings

Reversing text is a classic interview question, but the implementation matters:

// Naive — breaks emoji and surrogate pairs
"hello".split("").reverse().join("");

// Unicode-aware — handles most emoji correctly
[..."hello 👋"].reverse().join("");

The spread operator iterates over Unicode code points (not UTF-16 code units), so it correctly handles single emoji. However, it still breaks ZWJ sequences (family emoji, skin tones) and combining characters (letters with accents). For production use, consider a grapheme-aware library.

Python handles this more gracefully:

text[::-1]  # slice reversal — works on code points

For quick string reversal, use our Text Reverser.

Building Text Pipelines

Real-world text processing often chains multiple operations. Here is a pipeline that cleans up a messy list of email addresses:

const cleaned = rawText
  .split("\n")                     // split into lines
  .map(line => line.trim())        // trim whitespace
  .filter(line => line.length > 0) // remove empty lines
  .map(line => line.toLowerCase()) // normalize case
  .filter(line => line.includes("@"))  // basic email filter
  .sort((a, b) => a.localeCompare(b))  // sort alphabetically
  .filter((v, i, a) => a[i - 1] !== v); // remove adjacent duplicates

Terminal equivalent:

cat emails.txt | tr -s ' ' | sed '/^$/d' | tr 'A-Z' 'a-z' | grep '@' | sort -u

The Unix philosophy of small composable tools applies to text processing in any language. Each step does one thing. Chain them together for complex transformations.

Performance Tips

• Large files (>100 MB): Use streams instead of loading everything into memory. Node.js readline or Python open() with line-by-line iteration.
• Repeated operations: Compile regex patterns once outside the loop (new RegExp() in JS, re.compile() in Python).
• Sorting: JavaScript’s built-in .sort() uses Timsort (O(n log n)). For specialized sorts on large datasets, consider radix sort for integers or bucket sort for known distributions.
• Deduplication of huge datasets: A Set in JS or set in Python works for millions of strings. For billions, consider probabilistic data structures like Bloom filters.

Summary

For more text manipulation techniques, check out our Regex Cheat Sheet with Examples and the URL Slug Best Practices guide.

Further Reading

• MDN: String methods — comprehensive reference for JavaScript string operations
• Python: String methods — official Python docs on built-in string methods
• GNU Coreutils: Text utilities — sort, uniq, wc, tr, cut, and more
• Unicode Text Segmentation — the Unicode standard for splitting text into grapheme clusters, words, and sentences