Utilized Python Chronicles: A Light Intro to Pydantic | by Ilija Lazarevic | Jul, 2024

What about default values and argument extractions?

from pydantic import validate_call
@validate_call(validate_return=True)
def add(*args: int, a: int, b: int = 4) -> int:
return str(sum(args) + a + b)
# ----
add(4,3,4)
> ValidationError: 1 validation error for add
a
Lacking required key phrase solely argument [type=missing_keyword_only_argument, input_value=ArgsKwargs((4, 3, 4)), input_type=ArgsKwargs]
For additional info go to <https://errors.pydantic.dev/2.5/v/missing_keyword_only_argument>
# ----
add(4, 3, 4, a=3)
> 18
# ----
@validate_call
def add(*args: int, a: int, b: int = 4) -> int:
return str(sum(args) + a + b)
# ---- 
add(4, 3, 4, a=3)
> '18'

Takeaways from this instance:

• You possibly can annotate the kind of the variable variety of arguments declaration (*args).
• Default values are nonetheless an possibility, even in case you are annotating variable information varieties.
• validate_call accepts validate_return argument, which makes operate return worth validation as nicely. Information kind coercion can be utilized on this case. validate_return is ready to False by default. Whether it is left as it’s, the operate might not return what is said in kind hinting.

What about if you wish to validate the information kind but in addition constrain the values that variable can take? Instance:

from pydantic import validate_call, Area
from typing import Annotated 
type_age = Annotated[int, Field(lt=120)]
@validate_call(validate_return=True)
def add(age_one: int, age_two: type_age) -> int:
return age_one + age_two
add(3, 300)
> ValidationError: 1 validation error for add
1
Enter needs to be lower than 120 [type=less_than, input_value=200, input_type=int]
For additional info go to <https://errors.pydantic.dev/2.5/v/less_than>

This instance reveals:

• You should utilize Annotated and pydantic.Area to not solely validate information kind but in addition add metadata that Pydantic makes use of to constrain variable values and codecs.
• ValidationError is but once more very verbose about what was mistaken with our operate name. This may be actually useful.

Right here is yet one more instance of how one can each validate and constrain variable values. We’ll simulate a payload (dictionary) that you just wish to course of in your operate after it has been validated:

from pydantic import HttpUrl, PastDate
from pydantic import Area
from pydantic import validate_call
from typing import Annotated
Title = Annotated[str, Field(min_length=2, max_length=15)]
@validate_call(validate_return=True)
def process_payload(url: HttpUrl, title: Title, birth_date: PastDate) -> str:
return f'{title=}, {birth_date=}'
# ----
payload = {
'url': 'httpss://instance.com',
'title': 'J',
'birth_date': '2024-12-12'
}
process_payload(**payload)
> ValidationError: 3 validation errors for process_payload
url
URL scheme needs to be 'http' or 'https' [type=url_scheme, input_value='httpss://example.com', input_type=str]
For additional info go to <https://errors.pydantic.dev/2.5/v/url_scheme>
title
String ought to have not less than 2 characters [type=string_too_short, input_value='J', input_type=str]
For additional info go to <https://errors.pydantic.dev/2.5/v/string_too_short>
birth_date
Date needs to be up to now [type=date_past, input_value='2024-12-12', input_type=str]
For additional info go to <https://errors.pydantic.dev/2.5/v/date_past>
# ----
payload = {
'url': '<https://instance.com>',
'title': 'Joe-1234567891011121314',
'birth_date': '2020-12-12'
}
process_payload(**payload)
> ValidationError: 1 validation error for process_payload
title
String ought to have at most 15 characters [type=string_too_long, input_value='Joe-1234567891011121314', input_type=str]
For additional info go to <https://errors.pydantic.dev/2.5/v/string_too_long>

This was the fundamentals of how you can validate operate arguments and their return worth.

Now, we are going to go to the second most essential means Pydantic can be utilized to validate and course of information: by defining fashions.

This half is extra fascinating for the needs of information processing, as you will notice.

To date, we now have used validate_call to brighten capabilities and specified operate arguments and their corresponding varieties and constraints.

Right here, we outline fashions by defining mannequin courses, the place we specify fields, their varieties, and constraints. That is similar to what we did beforehand. By defining a mannequin class that inherits from Pydantic BaseModel, we use a hidden mechanism that does the information validation, parsing, and serialization. What this offers us is the flexibility to create objects that conform to mannequin specs.

Right here is an instance:

from pydantic import Area
from pydantic import BaseModel
class Particular person(BaseModel):
title: str = Area(min_length=2, max_length=15)
age: int = Area(gt=0, lt=120)
# ----
john = Particular person(title='john', age=20)
> Particular person(title='john', age=20)
# ----
mike = Particular person(title='m', age=0)
> ValidationError: 2 validation errors for Particular person
title
String ought to have not less than 2 characters [type=string_too_short, input_value='j', input_type=str]
For additional info go to <https://errors.pydantic.dev/2.5/v/string_too_short>
age
Enter needs to be larger than 0 [type=greater_than, input_value=0, input_type=int]
For additional info go to <https://errors.pydantic.dev/2.5/v/greater_than>

You should utilize annotation right here as nicely, and it’s also possible to specify default values for fields. Let’s see one other instance:

from pydantic import Area
from pydantic import BaseModel
from typing import Annotated 
Title = Annotated[str, Field(min_length=2, max_length=15)]
Age = Annotated[int, Field(default=1, ge=0, le=120)]
class Particular person(BaseModel):
title: Title
age: Age
# ----
mike = Particular person(title='mike')
> Particular person(title='mike', age=1)

Issues get very fascinating when your use case will get a bit complicated. Bear in mind the payload that we outlined? I’ll outline one other, extra complicated construction that we’ll undergo and validate. To make it extra fascinating, let’s create a payload that we’ll use to question a service that acts as an middleman between us and LLM suppliers. Then we are going to validate it.

Right here is an instance:

from pydantic import Area
from pydantic import BaseModel
from pydantic import ConfigDict
from typing import Literal
from typing import Annotated
from enum import Enum
payload = {
"req_id": "take a look at",
"textual content": "It is a pattern textual content.",
"instruction": "embed",
"llm_provider": "openai",
"llm_params": {
"llm_temperature": 0,
"llm_model_name": "gpt4o"
},
"misc": "what"
}
ReqID = Annotated[str, Field(min_length=2, max_length=15)]
class LLMProviders(str, Enum):
OPENAI = 'openai'
CLAUDE = 'claude'
class LLMParams(BaseModel):
temperature: int = Area(validation_alias='llm_temperature', ge=0, le=1)
llm_name: str = Area(validation_alias='llm_model_name', 
serialization_alias='mannequin')
class Payload(BaseModel):
req_id: str = Area(exclude=True)
textual content: str = Area(min_length=5)
instruction: Literal['embed', 'chat']
llm_provider: LLMProviders
llm_params: LLMParams
# model_config = ConfigDict(use_enum_values=True)
# ----
validated_payload = Payload(**payload)
validated_payload
> Payload(req_id='take a look at', 
textual content='It is a pattern textual content.', 
instruction='embed', 
llm_provider=<LLMProviders.OPENAI: 'openai'>, 
llm_params=LLMParams(temperature=0, llm_name='gpt4o'))
# ----          
validated_payload.model_dump()
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': <LLMProviders.OPENAI: 'openai'>,
'llm_params': {'temperature': 0, 'llm_name': 'gpt4o'}}
# ----
validated_payload.model_dump(by_alias=True)
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': <LLMProviders.OPENAI: 'openai'>,
'llm_params': {'temperature': 0, 'mannequin': 'gpt4o'}}
# ----
# After including 
#     model_config = ConfigDict(use_enum_values=True)
# in Payload mannequin definition, you get
validated_payload.model_dump(by_alias=True)
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': 'openai',
'llm_params': {'temperature': 0, 'mannequin': 'gpt4o'}}

Among the essential insights from this elaborated instance are:

• You should utilize Enums or Literal to outline an inventory of particular values which might be anticipated.
• In case you wish to title a mannequin’s area in another way from the sphere title within the validated information, you should utilize validation_alias. It specifies the sphere title within the information being validated.
• serialization_alias is used when the mannequin’s inside area title will not be essentially the identical title you wish to use while you serialize the mannequin.
• Area may be excluded from serialization with exclude=True.
• Mannequin fields may be Pydantic fashions as nicely. The method of validation in that case is finished recursively. This half is actually superior, since Pydantic does the job of going into depth whereas validating nested constructions.
• Fields that aren’t taken under consideration within the mannequin definition should not parsed.

Right here I’ll present you the snippets of code that present the place and the way you should utilize Pydantic in your day-to-day duties.

Say you have got information it is advisable to validate and course of. It may be saved in CSV, Parquet information, or, for instance, in a NoSQL database within the type of a doc. Let’s take the instance of a CSV file, and let’s say you wish to course of its content material.

Right here is the CSV file (take a look at.csv) instance:

title,age,bank_account
johnny,0,20
matt,10,0
abraham,100,100000
mary,15,15
linda,130,100000

And right here is how it’s validated and parsed:

from pydantic import BaseModel
from pydantic import Area 
from pydantic import field_validator
from pydantic import ValidationInfo
from typing import Listing
import csv
FILE_NAME = 'take a look at.csv'
class DataModel(BaseModel):
title: str = Area(min_length=2, max_length=15)
age: int = Area(ge=1, le=120)
bank_account: float = Area(ge=0, default=0)
@field_validator('title')
@classmethod
def validate_name(cls, v: str, information: ValidationInfo) -> str:
return str(v).capitalize()
class ValidatedModels(BaseModel):
validated: Listing[DataModel]
validated_rows = []
with open(FILE_NAME, 'r') as f:
reader = csv.DictReader(f, delimiter=',')
for row in reader:
strive:
validated_rows.append(DataModel(**row))
besides ValidationError as ve:
# print out error
# disregard the document
print(f'{ve=}')
validated_rows
> [DataModel(name='Matt', age=10, bank_account=0.0),
DataModel(name='Abraham', age=100, bank_account=100000.0),
DataModel(name='Mary', age=15, bank_account=15.0)]
validated = ValidatedModels(validated=validated_rows)
validated.model_dump()
> {'validated': [{'name': 'Matt', 'age': 10, 'bank_account': 0.0},
{'name': 'Abraham', 'age': 100, 'bank_account': 100000.0},
{'name': 'Mary', 'age': 15, 'bank_account': 15.0}]}

FastAPI is already built-in with Pydantic, so this one goes to be very temporary. The way in which FastAPI handles requests is by passing them to a operate that handles the route. By passing this request to a operate, validation is carried out robotically. One thing just like validate_call that we talked about originally of this text.

Instance of app.py that’s used to run FastAPI-based service:

from fastapi import FastAPI
from pydantic import BaseModel, HttpUrl
class Request(BaseModel):
request_id: str
url: HttpUrl
app = FastAPI()
@app.submit("/search/by_url/")
async def create_item(req: Request):
return merchandise

Pydantic is a extremely highly effective library and has plenty of mechanisms for a mess of various use circumstances and edge circumstances as nicely. In the present day, I defined probably the most primary elements of how it is best to use it, and I’ll present references beneath for individuals who should not faint-hearted.

Go and discover. I’m positive it’s going to serve you nicely on totally different fronts.