Changelog
v0.1.1 — 30 June 2026
The marketplace lands as one unified surface: free public discovery and paid datasets share the same model, the same listing form, the same URL, and the same buy flow. Lower-friction for publishers, simpler mental model for buyers.
Marketplace
- One marketplace, free + paid.
/marketplaceis now a single index that shows every listed dataset with a clear price chip ("Free" / "$49" / "$19/mo"). Setting a price on the listing form flips a listing paid; clearing it makes it free again. No separate storefront URL. - Paid listings check out via Stripe — one-time charges or
recurring monthly/yearly subscriptions. Inline
price_datameans no pre-created Stripe Price per dataset to maintain. - Listing detail page (
/d/<slug>) renders the same template for both modes. Free → row preview + open download. Paid + no purchase → description, schema, table sizes, license terms + Buy CTA, with row values hidden until purchase. Paid + validmp_accesscookie → full preview + download. Public dataset slug stays the canonical URL — bookmarks survive a listing's price changes. - Access cookie (
mp_access, HttpOnly, SameSite=Lax, 30 days) is dropped when a buyer follows the receipt link. Subsequent visits to the dataset page see the gate open without re-auth. - Subscription lifecycle is wired:
customer.subscription.deleted/canceled/unpaidflips the matching DatasetPurchase tocanceled, revoking access at the next download attempt. - Listing form gains optional
price_cents,billing_interval(one_time / month / year), andlicense_termsfields. Validation rejects sub-$0.50 (Stripe floor) and >$9999.
SEO / public surfaces
/robots.txt+/sitemap.xmlship — allow public, disallow dashboard/admin/auth/api/buy/checkout/billing. Sitemap only lists listed datasets (rung 3 of the privacy ladder), honoring the publisher's choice not to be indexed.- OG/Twitter meta tags on every
/d/<slug>so dataset shares to Slack, Twitter, LinkedIn, Discord render with title + hook. ProxyFixmiddleware (x_proto=1, x_host=1) so_external=TrueURLs producehttps://pipes.run/...behind Cloud Run's TLS- terminating frontend. Affects OG canonicals, sitemap entries, and magic-link emails.
Marketing site
- Marketplace surfaced in landing-page Product footer + the shared footer on every other page. (Header promotion deferred until ~10 listings exist.)
- Landing-page FAQ entry explaining how the public marketplace coexists with the local-first promise (opt-in publishing, two confirmed clicks before anything is visible).
- Privacy §2.4 + Terms §11 added covering marketplace publishing: what becomes public, what we deliberately don't expose, UGC license buyers grant us to host, publisher warranties, DMCA takedown flow, no-warranty disclaimer on third-party datasets. "Last updated" bump to 30 June 2026. Banners still mark these as draft pending counsel review.
Reliability
dataset_published_at TIMESTAMPcolumn added tocloud_pipelines(wasDATETIMEin an earlier rev — Postgres has no DATETIME type, caused a boot crash that's now fixed).view_countactually increments on each/d/<slug>view (atomic SQL UPDATE, not read-modify-write). The model promised it; now the route delivers.- Removed duplicate
dataset_public+dataset_slugcolumn definitions fromCloudPipeline.
v0.1.0 — 25 May 2026
The first public release. Everything below ships in v0.1.0; later releases will be incremental updates from here.
Desktop app
- Chat with the web, save as pipeline. Three search verticals (web, news, past day/week/month/year), DuckDuckGo + Bing + Brave with automatic fallback. Local LLM query rewriting turns chat-style prompts ("find AI funding announcements from this week") into clean keyword queries before hitting the search engine.
- 10 LLM providers. Anthropic, OpenAI, Gemini natively; Perplexity, Groq, Mistral, Together, OpenRouter, Cohere via LiteLLM; Ollama for local models with auto-detection.
- Multi-step transforms with
{row}/{row.field}/{field}placeholders so each step sees the prior outputs. - Three destinations: SQLite (default), Postgres, DuckDB.
- Write modes: append, replace, upsert (with compound dedupe keys).
- Schedule in plain English ("every weekday at 9am") or cron.
- Sources: web search, crawl, HTTP/JSON, CSV, Excel, Google Sheets (OAuth), SQL query against any of your databases.
- Workspace backup — every pipeline + provider as one YAML file. API keys are intentionally excluded.
- Run cancellation, failure notifications, pipeline dependencies, run history modal, conversation history drawer all in.
- First-run health check detects missing crawler + unconfigured LLM providers and offers one-click fixes.
- Auto-update banner + license validation with 7-day offline grace.
- Signed + notarized macOS DMG — no Gatekeeper warnings, installs like any Mac app.
Cloud
- Marketing site (landing, pricing, privacy, terms) with hero chat mockup, FAQ, comparison table.
- Magic-link sign-in via Cloudflare Email Worker + MailChannels.
- Stripe Checkout for Desktop ($99 one-time) + Cloud subscription ($19/month). Webhook-verified license minting; idempotent retries.
- Customer dashboard with license, download, cloud-pipeline list, recent runs.
- Desktop ↔ cloud sync API (
/api/v1/pipelines,/api/v1/runs) for pushing pipelines to always-on cloud infrastructure. - Mock cloud executor runs a background thread that picks pending
runs and processes them — to be replaced with the real
pipes.runner.run_pipelinefor real cloud execution before launch.
Admin / ops
- Admin user view (
/admin) for support: search users, see their licenses + seats, reset seats, comp a license, revoke a license, resend the receipt email. Every action requires a reason field that gets persisted to the audit log. - Audit log of every admin action — actor, target, action, reason, timestamp.