Python String Validation: Unicode Predicates and Input Pipelines
Introduction
Python string validation requires both a shape rule and a conversion policy. "²".isdigit() is true, but int("²") fails; "-5".isdigit() is false, but int("-5") succeeds. Treating those operations as interchangeable can reject useful data and still admit failures.
This lesson completes objective 3.3 of PCAP-31-03 Strings (18% of the score, 8 of 40 items), connecting Unicode representation with exception handling. You will distinguish character predicates, enforce ASCII rules when required, and build a staged input pipeline that reports invalid fields.
Learning objectives
After this lesson you can:
- Use
isalpha(),isdigit(),isdecimal(),isnumeric(),isalnum(),isspace(),isupper(),islower(),istitle()andisascii()and predict their results. - Explain the behaviour of these tests on the empty string and on non-ASCII characters.
- Distinguish a test such as
isdigit()from a conversion such asint(). - Chain methods and trace each intermediate state.
- Convert text to numbers with
int()andfloat()and handleValueError. - Design a validation pipeline for identifiers and quantities.
The isxxx() family
Each predicate returns a bool, but their rules differ. The alphabetic and numeric tests require at least one character. Case tests examine cased characters; identifier and title tests have structural rules. isascii() and isprintable() also accept the empty string.
| Method | Condition for True |
|---|---|
isalpha() |
every character is a letter |
isdigit() |
every character is a digit (including superscript digits) |
isdecimal() |
every character is a decimal digit usable to form numbers |
isnumeric() |
every character is numeric (digits, fractions, roman numerals...) |
isalnum() |
every character is a letter or a numeric character |
isspace() |
every character is whitespace |
isupper() |
it has at least one cased character and all cased characters are uppercase |
islower() |
it has at least one cased character and all cased characters are lowercase |
istitle() |
at least one cased character; cased runs start uppercase/titlecase and continue lowercase |
isidentifier() |
it has valid identifier syntax; reserved keywords need a separate check |
isascii() |
every code point is below 128, including an empty string |
isprintable() |
every character is printable under Python’s rule, including an empty string |
print("Python".isalpha()) # True
print("Python3".isalpha()) # False
print("2026".isdigit()) # True
print("20.26".isdigit()) # False, the dot is not a digit
print("-5".isdigit()) # False, the minus sign is not a digit
print("a1".isalnum()) # True
print("a 1".isalnum()) # False, the space is not alphanumeric
print(" \t\n".isspace()) # True
print("Hello World".istitle())# True
print("hello World".istitle())# FalseEmpty strings
The alphabetic, numeric, whitespace, identifier, and case tests above return False for "". isascii() and isprintable() return True because no character breaks their all-characters rule:
print("".isalpha(), "".isdigit(), "".isalnum(), "".isspace()) # False False False False
print("".isupper(), "".islower()) # False False
print("".isascii(), "".isprintable()) # True TrueThis matters when you validate user input. A check such as if not value.isdigit(): reject correctly rejects an empty field, while a hand-written loop all(c.isdigit() for c in value) returns True for an empty field, because all of nothing is true. The two versions are not equivalent.
Cased tests ignore other characters
isupper() requires at least one cased character, and the characters without case do not matter:
print("ABC".isupper()) # True
print("ABC123".isupper()) # True, the digits have no case
print("123".isupper()) # False, no cased character
print("AbC".isupper()) # False
print("abc-1".islower()) # True
print("Abc".islower()) # FalseNon-ASCII behaviour: isdigit, isdecimal, isnumeric
The three numeric tests differ for characters outside ASCII. Unicode contains many digit characters: Arabic-Indic digits, superscripts, fractions. The tests form a hierarchy, from the narrowest to the broadest:
| Character | isdecimal() |
isdigit() |
isnumeric() |
|---|---|---|---|
"7" (ASCII digit) |
True | True | True |
"٣" (Arabic-Indic three) |
True | True | True |
"²" (superscript two) |
False | True | True |
"½" (one half) |
False | False | True |
"Ⅳ" (Roman numeral four) |
False | False | True |
"x" |
False | False | False |
Every decimal is a digit and every digit is numeric. The narrowest test, isdecimal(), accepts characters that can form a base-10 number. This is why isdigit() is not a guarantee that int() will succeed:
print("²".isdigit()) # True
print(int("٣")) # 3, Python accepts decimal characters from other scripts
print(int("²")) # ValueError: invalid literal for int() with base 10: '²'
print("٣٤".isdecimal()) # True
print(int("٣٤") + 1) # 35Review the difference between characters and encoded bytes if a displayed symbol contains multiple code points. If you must accept only the ASCII digits 0 to 9, combine two checks, for example s.isascii() and s.isdigit(), or s.isdecimal() and s.isascii(). Testing isdigit() alone also accepts superscripts and digits of other scripts, which may be unwanted in an identifier or in a database column that expects ASCII.
Letters behave in the same way: "é".isalpha() is True and "é".isascii() is False, so isalpha() does not mean "a to z":
print("café".isalpha()) # True
print("café".isascii()) # False
print("naïve".isalnum()) # True
print("日本語".isalpha()) # TrueWorked example: validating a catalog identifier
Problem. Catalog identifiers have the form AB1234: exactly two ASCII uppercase letters followed by exactly four ASCII digits. Write valid_id(text) and trace it for "AB1234", "ab1234", "AB١٢٣٤" (Arabic-Indic digits), "ÉB1234" and "".
Step 1: break down the rule. Length 6; the first two characters are ASCII uppercase letters; the last four are ASCII digits.
Step 2: write the checks. The tests on the whole string isalpha()/isdigit() are too permissive, so combine them with isascii().
def valid_id(text):
if len(text) != 6 or not text.isascii():
return False
return text[:2].isalpha() and text[:2].isupper() and text[2:].isdigit()Step 3: trace each input.
"AB1234": length 6, ASCII;"AB"is alphabetic and uppercase;"1234"is made of digits. ResultTrue."ab1234": the first part is alphabetic but not uppercase. ResultFalse."AB١٢٣٤": the length is 6, but the string is not ASCII (the Arabic-Indic digits are above 127), so the first test fails. ResultFalse."ÉB1234": the letterÉis alphabetic and uppercase, but it is not ASCII. ResultFalse."": the length is 0. ResultFalse.
Solution. Only the first input is accepted.
Why other designs are worse. Without the isascii() guard, the third and fourth inputs would pass, because isdigit() and isalpha() accept Unicode characters. Without the length test, "AB12345" would pass. Checking only text.isalnum() would accept "12AB34".
Identifier syntax is not keyword validation
"class".isidentifier() is True, but class is reserved. If validating a variable name, combine text.isidentifier() with not keyword.iskeyword(text) after importing keyword. Similarly, istitle() follows runs of cased characters rather than linguistic word boundaries: punctuation or digits can start a new run.
Tests versus conversions
A test returns a boolean, a conversion returns a new value or raises an exception. The functions int() and float() convert text to numbers and raise ValueError when the text is not a valid number. They are more flexible than isdigit():
print(int("42")) # 42
print(int(" 42 ")) # 42, surrounding whitespace is allowed
print(int("+7")) # 7
print(int("-7")) # -7
print(int("1_000")) # 1000, underscores between digits are allowed
print(int("3.5")) # ValueError
print(int("")) # ValueError
print(int("12a")) # ValueError
print(float("3.5")) # 3.5
print(float("1e3")) # 1000.0
print(float(" -0.5 ")) # -0.5
print(int("ff", 16)) # 255
print(int(3.9)) # 3, a float is truncated; no exceptionTwo consequences follow:
"-5".isdigit()isFalsebutint("-5")works. A test cannot replace a conversion for signed numbers.int("3.5")fails even thoughfloat("3.5")works. If truncation is an explicit requirement,int(float("3.5"))gives 3. This is not validation of an integer field: it discards a fraction, and floating-point conversion can lose precision for large values.
There are two classic styles. Look before you leap tests first, then converts. Easier to ask forgiveness converts directly and catches the exception. Choose the policy first. Converting directly is convenient when Python’s integer grammar is the accepted format, including signs, spaces, and underscores. A strict identifier or fixed-format field needs its own checks:
def to_quantity(text):
try:
value = int(text)
except ValueError:
return None
return value if value >= 0 else None
print(to_quantity(" 12 ")) # 12
print(to_quantity("-3")) # None
print(to_quantity("abc")) # None
print(to_quantity("")) # NoneNote that the try block contains only the conversion. Placing more code inside it could hide unrelated ValueError exceptions.
If the argument is not a string at all (for example None), int() raises TypeError, which is not a ValueError. An API that deliberately accepts ordinary strings or None can handle both expected errors. Arbitrary objects can run custom conversion methods and raise other exceptions; catching these two types is not a guarantee for every object.
Method chains and intermediate states
String-returning methods such as strip, lower, and replace can be chained from left to right. Other methods return integers, lists, tuples, or booleans, so check the intermediate type. Each call works on the result of the previous one, so you trace a chain by writing the intermediate value after every dot.
raw = " PX-2048 \n"
print(raw.strip().lower().replace("-", "")) # px2048State after each step:
| Step | Expression | Value |
|---|---|---|
| 0 | raw |
" PX-2048 \n" |
| 1 | .strip() |
"PX-2048" |
| 2 | .lower() |
"px-2048" |
| 3 | .replace("-", "") |
"px2048" |
The order matters. Consider " Hello World ".strip().split() and " Hello World ".split().strip(). The first gives ['Hello', 'World']. The second raises AttributeError, because split() returns a list and a list has no strip method. A chain is valid only if each result has the method used next.
Other useful transformations are lower(), upper(), capitalize() (first character titlecased, the rest lowercase), title(), swapcase() and replace(old, new[, count]):
print("pyTHON exAM".capitalize()) # Python exam
print("pyTHON exAM".title()) # Python Exam
print("pyTHON exAM".swapcase()) # PYthon EXam
print("a-b-c-d".replace("-", "+", 2))# a+b+c-d
print("Straße".upper()) # STRASSE
print("Straße".casefold()) # strasseThe last two lines show that case conversion on Unicode text may change the length: the German letter ß becomes SS. For case-insensitive comparison of international text, casefold() is the recommended method.
Worked example: a three-stage pipeline
Problem. A form gives a quantity and a country code as text. Build a function clean(qty, country) that returns (int, str): the quantity must be a non-negative integer, the country code must be exactly two ASCII letters and is stored in uppercase. Invalid values raise ValueError with a message. Trace clean(" 15 ", "fr") and clean("1.5", "FR") and clean("7", "é1").
def clean(qty, country):
code = country.strip()
if not (len(code) == 2 and code.isascii() and code.isalpha()):
raise ValueError("bad country: " + repr(country))
try:
number = int(qty)
except ValueError:
raise ValueError("bad quantity: " + repr(qty)) from None
if number < 0:
raise ValueError("negative quantity")
return number, code.upper()Step 1: clean(" 15 ", "fr"). The country fr has length 2, is ASCII and alphabetic, so the check passes. int(" 15 ") is 15 because whitespace is tolerated. The function returns (15, 'FR').
Step 2: clean("1.5", "FR"). The country is accepted. int("1.5") raises ValueError, which the except clause translates into a message that includes the original text with repr(): bad quantity: '1.5'.
Step 3: clean("7", "é1"). The length is 2, but isascii() is False, so the country check fails first and raises bad country: 'é1'. The quantity is never examined.
Solution. The three calls return (15, 'FR'), raise bad quantity: '1.5' and raise bad country: 'é1'.
Why this design. The country check uses a test because the rule is a shape; the quantity uses a conversion because Python already defines what a valid integer is. The handler raises a new ValueError with a field-specific message. from None hides the implicit context in the default traceback display; the original exception remains inspectable. Use custom validation exceptions when callers need different exception types for different fields. The country check validates shape, not membership in an official country-code registry. Each stage has one responsibility, and each failure produces a clear message.
Choosing the right test
| Question | Test or conversion | Caution |
|---|---|---|
| only ASCII digits? | s.isascii() and s.isdigit() |
isdigit alone accepts ² |
any decimal digits to build an int? |
s.isdecimal() |
other scripts are accepted |
| any numeric text (fractions)? | s.isnumeric() |
int() may still fail |
| a signed or spaced number? | try: int(s) |
tests reject -5 |
| only letters? | s.isalpha() |
accepts é and 日 |
| blank or empty? | not s.strip() |
isspace is False for "" |
Tracing drill
values = ["42", " 7", "-3", "²", "", "٣", "4.0", "1_0"]
tests = [v.isdigit() for v in values]
print(tests)
ok = []
for v in values:
try:
ok.append(int(v))
except ValueError:
ok.append(None)
print(ok)
print(sum(n for n in ok if n is not None))isdigit()for each value:"42"isTrue;" 7"contains a space,False;"-3"has a sign,False;"²"is a superscript digit,True;""is empty,False;"٣"is an Arabic-Indic digit,True;"4.0"has a dot,False;"1_0"has an underscore,False. The list is[True, False, False, True, False, True, False, False].int()accepts"42"(42)," 7"(7),"-3"(-3). It rejects"²"(ValueError),""and"4.0". It accepts"٣"(3) and"1_0"(10). The list is[42, 7, -3, None, None, 3, None, 10].- The sum of the non-
Nonevalues is 42 + 7 - 3 + 3 + 10 = 59.
The drill shows that tests and conversions disagree in both directions: "²" passes the test but fails the conversion, while " 7" and "-3" fail the test but pass the conversion.
Mini case study: importing stock counts
A shop imports a text file with one signed inventory adjustment per line, including positive receipts and negative corrections. The first version of the import is:
total = 0
for line in lines:
if line.isdigit():
total += int(line)The team notices that totals are too low and that a rare crash occurs on some lines.
Task. Explain both symptoms and propose a corrected loop.
Solution.
- Totals too low. Lines that carry a trailing newline (
"12\n") or spaces failisdigit(), so they are skipped silently. Lines with a plus sign are also skipped, and a legitimate negative adjustment ("-3") is lost. The error is silent, which is the worst kind. - Rare crash. A line containing a superscript digit such as
"²"passesisdigit()and then fails inint()withValueError. - Corrected loop. Convert directly, catch the error, and record problems instead of ignoring them.
total = 0
rejected = []
for number, line in enumerate(lines, start=1):
text = line.strip()
if not text:
continue
try:
total += int(text)
except ValueError:
rejected.append((number, line))The loop deliberately skips blank lines and records failed conversions with line numbers. Valid signed adjustments contribute to the total. CPython’s configurable decimal conversion length limit can reject very long digit strings with ValueError, even when isdecimal() is true; bound field length according to the application’s needs.
Common pitfalls and exam tips
- Empty-string behavior depends on the predicate.
isascii()andisprintable()accept""; alphabetic, numeric, whitespace, and case tests reject it. isdigit()is not the same as "convertible byint()":"²"passes the test and fails the conversion;"-5"and" 5"fail the test but convert.isalpha()accepts non-ASCII letters. Combine withisascii()for ASCII letters, and test case if onlyatozis allowed.isupper()ignores characters without case but needs at least one cased character.int("3.5")raisesValueError;int(3.5)returns 3.- A chain needs the right type at each step:
split()returns a list. ß.upper()isSS: case conversion can change length.- Catch only the exceptions you expect and keep the
tryblock small.
Exam tip. For any
isxxx()question, first check for an empty string, then check each character one by one against the rule, remembering that Unicode digits and letters count. For anyint()question, ask whether the text is a clean integer literal (optional sign, digits, optional underscores, optional surrounding whitespace).
Key terms
- Predicate: a method that returns a boolean, such as
isalpha(). - Decimal / digit / numeric: three nested Unicode categories of number characters.
- Input cleanup: applying an explicit policy such as stripping surrounding whitespace. Unicode normalization is the separate task of reconciling equivalent code-point sequences.
- Method chain: several method calls joined by dots.
- LBYL / EAFP: test first versus try and catch.
casefold(): aggressive lowercase conversion for comparisons.
Summary checklist
- I can predict each
isxxx()result, including on"". - I know why
isdigit(),isdecimal()andisnumeric()differ. - I know that
isalpha()andisdigit()accept non-ASCII characters. - I can convert with
int()andfloat()and catchValueError. - I can trace a method chain by writing every intermediate value.
- I can design a pipeline that validates, converts and reports errors.
- I can explain why a test and a conversion may disagree.
Official sources
- PCAP-31-03 exam syllabus, objectives 3.1, 3.3 and 2.1: https://pythoninstitute.org/pcap-31-03-exam-syllabus (opens in a new tab)
- String methods: https://docs.python.org/3.14/library/stdtypes.html#string-methods (opens in a new tab)
- Built-in types and numeric parsing (
int,float): https://docs.python.org/3.14/library/functions.html#int (opens in a new tab) - Unicode HOWTO: https://docs.python.org/3.14/howto/unicode.html (opens in a new tab)
