Logging into websites every single run is the fastest way to:
🚫 trigger anti-bot systems
🧩 face endless CAPTCHAs
🔒 get accounts locked
🐌 slow down your scraper
The correct pattern is simple:
Login once → persist browser profile → reuse it forever (until expiration).
In this article I will show how this works in a real project:
👉 https://github.com/AmaLS367/parts_info_collector
This project automates data extraction from Gemini’s web UI and keeps authentication between runs using a persistent browser profile.
🧠 The Core Idea
Instead of exporting cookies manually, Playwright can launch Chromium with a persistent user profile directory.
That directory stores:
cookies 🍪
localStorage
IndexedDB
login tokens
session metadata
Once created, it becomes your reusable authenticated identity.
📂 How the Project Does It
In parts_info_collector, authentication is handled via:
a user-data/ directory
first interactive run via first_start.bat
manual login to Gemini
all next runs reuse the same browser profile
So the flow is:
1️⃣ Run once with UI
2️⃣ Log in manually
3️⃣ Browser profile is saved
4️⃣ Next runs are headless and already authenticated
No repeated login.
No constant CAPTCHA hell 😌
🔐 Persistent Context in Playwright
This is the key API:
browser_context = playwright.chromium.launch_persistent_context(
user_data_dir="user-data",
headless=True
)
That single folder is everything.
If it exists, Playwright loads it.
If not, you run in visible mode and authenticate.
🧑💻 First Run: Create the Session
In the project, the first launch happens with UI enabled so you can log in:
browser_context = playwright.chromium.launch_persistent_context(
user_data_dir="user-data",
headless=False
)
You open Gemini, authenticate manually, and close the browser.
From that moment:
📁 user-data/ contains your session.
♻️ All Next Runs: Fully Headless
Subsequent executions simply reuse the same folder:
browser_context = playwright.chromium.launch_persistent_context(
user_data_dir="user-data",
headless=True
)
You are already logged in.
No forms.
No credentials.
No redirects.
⚠️ What Happens When the Session Expires?
Eventually cookies die.
The project handles this operationally:
you delete user-data/
or rerun first_start.bat
login again
profile is recreated
Simple, manual, and reliable.
You can extend this with:
detecting redirect to /login
checking DOM markers
auto-relogin logic
alerting
🛡️ Why Persistent Profiles Beat Cookie Dumps
Using a persistent profile directory is stronger than just exporting cookies:
✅ keeps IndexedDB tokens
✅ survives browser restarts
✅ mimics real user Chrome profile
✅ less suspicious than scripted logins
✅ perfect for long-running monitors
🚀 When You Should Use This Pattern
This approach is ideal for:
📊 price monitors
🤖 automation bots
🧠 AI web scrapers
📈 background workers
🔁 periodic collectors
Anything that runs for weeks.
🔗 Real Repository
Full project:
👉 https://github.com/AmaLS367/parts_info_collector
🧾 TL;DR
✅ use launch_persistent_context
✅ store profile in user-data/
✅ login once manually
✅ reuse forever
✅ refresh only when expired
Top comments (0)