🧾 QiAlly Document Services — MIBOR Member Directory Capture & Automation
Client: [Client Name]
Prepared by: QiAlly Document Services
Consultant: Cody Rice-Velasquez
Date: November 2025
1. Project Goal
Extract approximately 8,000 member listings (≈ 571 pages) from the MIBOR Member Directory with minimal manual effort, producing durable artifacts (HTML/text/screenshots) per page for later parsing and structured analysis.
Deliver a clean, automatable process that preserves login sessions, mitigates timeouts, and yields exportable CSV/JSON datasets for business intelligence and CRM ingestion.
2. Work Performed (Chronological Summary)
Step 1 — Site Access & Analysis
- Logged into
member.mibor.comand confirmed listings appear in a card-based directory with server-side pagination. - Verified directory uses DotNetNuke (DNN) + Telerik RadListView, enforcing 20 results per page.
Step 2 — Manual Capture
- Captured Pages 1–30 with full-page screenshots.
- Confirmed session resets occur around Page 31, requiring automation or checkpointing.
Step 3 — Deliverables Clarified
- Output formats selected: HTML/Text or Screenshots per page.
- Progress logged in
mibor_progress_up_to_page30.md.
Step 4 — Technical Limitations
- No direct page-jump feature; session resets after prolonged pagination.
- Manual capture alone would require ~570 page navigations.
3. Automation & Engineering Plan
- Analyzed contact-ID structure (non-sequential GUIDs).
- Confirmed server-driven pagination and authentication blocks.
- Developed browser-automation script with pause/resume, cookie persistence, and randomized human-like timing.
- Validated stability through 30-page runs; average ≈ 16 seconds per page.
Outputs
| File | Description |
|---|---|
scrape_mibor_cdp.js | Final automation script |
mibor_members_full.csv/json | Structured dataset |
mibor_pages/ | Saved HTML/screenshot pages |
checkpoint.json | Resume state |
.pw-profile/ | Persistent session profile |
4. Research & Testing (Non-Billable)
| Category | Description | Hours | Billable |
|---|---|---|---|
| Bot / Agent Testing | Explored autonomous agents, Notion API, browser automation | 1.5 h | ❌ |
| Framework Comparison | Selenium vs Playwright vs Puppeteer | 1 h | ❌ |
| Session Experiments | Cookie persistence & CDP connections | 0.5 h | ❌ |
| Pagination Research | Timeout and session-life testing | 0.5 h | ❌ |
Total Non-Billable Research: 3.5 hours
5. Billable Work & Pricing
| Description | Rate | Hours / Basis | Amount |
|---|---|---|---|
| Initial Research & Planning Fee (Setup + Testing) | flat | — | $50 USD |
| Automation Build & Verification | $40 /hr | 5 h | $200 USD |
| Packaging & Deliverables (CSV + JSON exports) | flat | — | $50 USD |
| Total Project Cost | $300 USD |
Payment Terms:
- $50 Research Fee due upon acceptance of proposal.
- Remaining $250 balance due upon final delivery of data package.
6. Technical Notes
- Estimated 14 records per page × 571 pages ≈ 8 K records.
- SSO resets around Page 30 handled by checkpoint recovery.
- Automation uses lawful session replication for archival use only — no redistribution of directory data.
7. Next Steps
- Client confirms preferred output mode (PNG screenshots or HTML/Text).
- QiAlly runs automation to complete pages 1–571.
- Export to CSV/JSON and Notion-ready table.
- Optional cloud deployment available for unattended runs.
8. Acceptance
| Client Signature | ___________________________ |
| Date | ___________________________ |
9. Summary Reflection
QiAlly engineered a hybrid automation process merging human-supervised precision with AI-assisted browser capture. The system preserves authenticated sessions, scales to hundreds of pages, and delivers structured datasets suitable for business-intelligence tools.
This engagement provides the client a repeatable workflow and clean, portable data package for future use.
Prepared by QiAlly Document Services — Professional Data Preparation & Automation Solutions