"Loading Pydantic models from JSON without running out of memory"
#Pydantic
Si vous utilisez #Pydantic, et que vous êtes confronté·es à des problèmes d'allocation mémoire lors de chargement de #JSON volumineux, voici une exemple de comment contourner le problème, en utilisant la bibliothèque #Python ijson.
https://pythonspeed.com/articles/pydantic-json-memory/
🌖 從 JSON 加載 Pydantic 模型，避免記憶體不足
➤ 解決 Pydantic 解析大型 JSON 檔案的記憶體瓶頸
✤ https://pythonspeed.com/articles/pydantic-json-memory/
本文探討了使用 Pydantic 從大型 JSON 文件中載入數據時，記憶體使用量過高的問題。作者介紹了兩種降低記憶體佔用的方法：一是使用增量 JSON 解析器 ijson，二是利用 `dataclass` 搭配 `slots`，以更有效率的方式儲存物件。透過實測，作者展示了不同方法的記憶體使用量差異，並建議 Pydantic 開發者考慮在未來版本中優化記憶體管理。
+ 這篇文章很有幫助，我一直遇到 Pydantic 解析大型 JSON 的問題，現在知道可以嘗試 ijson 和 dataclass 了！
+ 作者提出的解決方案很實用，而且提供了具體的程式碼範例，方便直接應用。希望 Pydantic 未來能內建這些優化功能。
#Python效能 #Pydantic #JSON解析 #記憶體管理
🚀 Ah, the classic dilemma: how to load massive #JSON files into #Pydantic without summoning the out-of-memory demon. 😅 Clearly, the solution is to write yet another verbose article pretending it's rocket science, because, you know, common sense isn't enough. 🧠✨
https://pythonspeed.com/articles/pydantic-json-memory/ #Loading #OutOfMemory #JSONFiles #TechHumor #DataProcessing #HackerNews #ngated
Loading Pydantic models from JSON without running out of memory
https://pythonspeed.com/articles/pydantic-json-memory/
#HackerNews #Pydantic #JSON #Memory #Optimization #Memory #Management #Python #Programming
What does #pydantic have to do with #AI now!? And which version to use anyway? pydantic, pydantic2, or pydantic3? My distribution only has python3-pydantic. #Python
Did some benchmarking of how fast #Python (3.13) w/ async #httpx and #Pydantic could query a JSON API vs. #GoLang (both including unmarshalling to native types and some regex validation).
While Python could do about 2.5k requests per second against localhost when running singlecore, Go did 10k.
However, not only is Go "only" 4× as fast, these differences quickly shrink to 30 % when you introduce an actual network between the hosts, and become irrelevant with rate limiting & DDoS protection.
the biggest things i need ai to do for me is to have a high initial elo ranking but also be trainable to scan all local docs and then also bring in a lots of real time data and open datasets 24/7, display results on series of dashboards #rag #pydantic #yacy #httrack #cached version #best stacks #free for commercial use #competitive intel #tailored data
Looking for some examples of people using Pydantic in bioinformatics. First relevant result is... oh my own conference talk. How unhelpful.
Как я строю удобную инфраструктуру вокруг Python-проектов: линтеры, Poetry, CI/CD и Docker
Poetry вместо pip, Ruff вместо flake8, FastAPI вместо ручной документации. Что реально упрощает жизнь Python-разработчику сегодня — на примерах и с реальными конфигами.
https://habr.com/ru/articles/904844/
#python #fastapi #pydantic #docker #poetry #infrastructure #devops #ruff #управление_разработкой #precommit
Pydantic Releases Sandboxed Python Execution Server for AI Agents via Model Context Protocol
#AI #Pydantic #PydanticAI #MCP #ModelContextProtocol #Python #LLMs #AgenticAI #OpenSource #DevTools #Pyodide #Deno #Sandboxing #AISecurity #AIIntegration
#pydantic, one of the most downloaded #python library of all times...
crossing billions of total download just recently...
in the recent years...
has changed from one of the most elegant docs to one of the most horrifying and useless documentations ever...
anything you want to find on their pages these days is a "good luck" with a sneaky smear on the face...
truly a garbage if I'm being completely honest.
Агенты в Pydantic AI от вызова LLM до MCP
Всем привет, сегодня я расскажу вам о том, как делать можно делать агентов с помощью Pydantic AI. Начнем с простых примеров в виде вызова LLM , а затем постепенно будем усложнять задачу, создавая более сложного агента. Начиная от простой валидации данных на выходе, заканчивая хайповым MCP протоколом.
Un altre descobriment en aquest camí ha estat #Pydantic. Com podia viure programàticament parlant sense Pydantic?
Pydantic és una biblioteca de Python que serveix per validar i gestionar dades amb tipus. Et permet definir models (com classes) on s’especifica el tipus de cada camp, i automàticament valida i converteix les dades rebudes. Integrat a la perfecció amb fastAPI.
API автотесты на Python с запуском на CI/CD и Allure отчетом
В этой статье разберём процесс написания API автотестов на Python, используя современные best practices. Кроме того, мы настроим их запуск в CI/CD с помощью GitHub Actions и сформируем Allure-отчёт с историей запусков. Цель статьи — не только показать, как писать качественные API автотесты, но и научить запускать их в CI/CD, получая удобные отчёты о результатах.
https://habr.com/ru/articles/895452/
#python #qa_automation #testing #github_actions #github_pages #allure #pytest #api_testing #pydantic #cicd
Pydantic for data validation and hydration
https://blog.narf.ssji.net/2025/03/30/pydantic-for-data-validation-and-hydration/
When using Python in any serious way, one quickly encounters Pydantic. While I’ve run into it in many frameworks, up to now, I never took the time to sit down and work out what it was doing, beyond something like “data models and validation based on type annotations”.
I have so far used dataclasses for my data models and data transfer objects (DTO). But it finally happened, and I needed finer validation on the data within my model. So I decided to take Pydantic for a spin myself, and see what it was doing for real.
tl;dr:
Pydantic _is_ data models and validation based on type annotations (amongst other things)
It’s a (almost) drop-in replacement for dataclasses
Type resolution and validation does add one order of magnitude over dataclasses, but it’s quite acceptable as a tradeoff for not having to write any bespoke validation code
Not only can it validate data, but it can also construct nested Pydantic objects, with support for polymorphism.
The pydantic.TypeAdapter exposes some lower-level mechanisms allowing to compose Pydantic behaviours as needed
[…]
![Screenshot of a terminal. Text editor: class Box(BaseModel): things: list[ThingOne | ThingTwo] bd = { "things": [ t1d, t2d] } b = Box(**bd) print(b) Shell: ~/pydantic$ python actual_ticdan.py type='one’ type=2 things=[ThingOne(type="one’), ThingTwo(type=2)]](https://files.mastodon.social/cache/media_attachments/files/114/249/703/046/056/366/original/46bf352d39717257.png)
Ara millor, la línia de temps "home" necessita un token vàlid però la línia de temps "public" (federada) no.
https://codeberg.org/spla/appy/commit/081cf661cc33d702cd0e53a74ab9030fbffd9311
He tingut que crear un nou model #Pydantic per els tuts de la línia de temps federada perquè no han de tenir "favourited", "reblogged", "muted", "bookmarked", "pinned" ni "filtered"
🚀 FastOpenAPI: because what the world really needed was YET ANOTHER tool generating docs for obscure #Python frameworks nobody uses 🎉. Forget simplicity, let's throw in #Pydantic v2 and call it #innovation. Bravo, mrfatalyst, truly pushing the boundaries of unnecessary complexity! 🙄
https://github.com/mr-fatalyst/fastopenapi #FastOpenAPI #DeveloperTools #Complexity #HackerNews #ngated
🤖 I wrote up some notes on Pydantic AI Agents yesterday and shared some projects that I'm using them in.
Client Info
Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst