Postcard Data Fetcher & Creator

These scripts were made in order to "re-send" postcards that originally failed preflight for RealMail (but can be used for any account to resend postcards). After grabbing the original PDFs from the 'content' S3 Bucket in Lob-Production, you will need to flatten transparencies, preflight, and perform other fixes on the PDFs to make them print-ready.

Once the PDFs are print ready, you can run these scripts to get the original payloads of the postcards, then create new postcards using the the payload data and preflighted PDFs.

This repository contains two scripts:

1. fetch_postcard_data.js - Fetches postcard data from the Lob API and extracts the original POST body fields to create new CSV files
2. create_postcards.js - Creates new postcards using CSV data with original POST body fields

Setup

1. Install dependencies:

   npm install

2. Create a .env file in the project directory with your Lob API key:

   LOB_API_KEY=your_lob_api_key_here
   

3. Make the script executable:

   chmod +x fetch_postcard_data.js

Usage

1. Fetch Postcard Data

Run the fetch script with a CSV file containing postcard IDs:

node fetch_postcard_data.js 3900_navaho_st_sw_postcards.csv

or

node fetch_postcard_data.js 7079_southwood_dr_jenison_postcards.csv

This will output a new CSV containing the original payload data of each of the original postcards, named like original_file_name + original_data.

Example:

node fetch_postcard_data.js 7079_southwood_dr_jenison_postcards.csv

will create 7079_southwood_dr_jenison_postcards_original_data.csv in the root directory.

2. Append 'front' and 'back' values to CSV

The _original_data.csv will contain the original payload that will be used to create new postcards.

Since the responses from the API don't include 'front' and 'back', it's necessary to paste those values into the corresponding columns in the CSV.

The PDFs to use should be located in the root directory. Paste their filename into the 'front' and 'back' columns.

Example:

id,front,back
psc_a65e310cdfc6a06b,front_original_preflighted.pdf,back_original_preflighted.pdf

This can be easily done by highlighting all rows, then using the shift + alt + i shortcut in VSCode to put a cursor at the end of each selected line. Then move the cursor to the appropriate comma-separated-value and paste the filename. Repeat this process for both front and back values.

TODO: Automate this by standardizing the .csv filename and the .pdf filenames. If the PDFs already exist in the directory, we can find them based on filename and automatically append them.

3. Create New Postcards

Run the create script with a CSV file containing original postcard data:

node create_postcards.js 174_bradley_drive_postcards_original_data.csv

or

node create_postcards.js 3900_navaho_st_sw_postcards_original_data.csv

Output

Fetch Script Output

The fetch script creates a new CSV file with the original data fields:

• Input: 3900_navaho_st_sw_postcards.csv
• Output: 3900_navaho_st_sw_postcards_original_data.csv

The output CSV contains the following columns:

• id: Postcard ID
• mail_type: Mail type (e.g., "usps_standard")
• from_name, from_address_line1, from_address_country, from_address_state, from_address_city, from_address_zip: From address fields
• to_name, to_address_line1, to_address_country, to_address_state, to_address_city, to_address_zip: To address fields
• description: Postcard description
• metadata_user, metadata_campaign_id, metadata_environment, metadata_client: Metadata fields
• size: Postcard size
• front, back: These will be empty. Here you will need to paste the filenames of the PDFs to use (they should be in the same root directory as this script)

Create Script Output

The create script generates two CSV files:

Success Log (*_success.csv):

• original_id: Original postcard ID from input CSV
• new_postcard_id: New postcard ID created by Lob API
• status: Postcard status (processed, etc.)
• mail_type, size, carrier: Postcard specifications
• from_name, from_address_line1, from_address_city, from_address_state, from_address_zip, from_address_country: From address details
• to_name, to_address_line1, to_address_city, to_address_state, to_address_zip, to_address_country: To address details
• description: Postcard description
• metadata_user, metadata_campaign_id, metadata_environment, metadata_client: Metadata fields
• date_created, date_modified, send_date, expected_delivery_date, completed_at: Timestamps
• url, raw_url: Postcard PDF URLs
• use_type, fsc, lob_credits_funding_status, is_creative_proof: Additional postcard details
• response_data: Full API response JSON

Error Log (*_errors.csv):

• original_id: Original postcard ID from input CSV
• status_code: HTTP status code
• error_message, error_details: Error information
• request_payload: Original request data that failed
• raw_response: Raw API error response
• from_name, to_name, description: Postcard details for reference

Features

• Handles API rate limiting with delays between requests (commented out, this script is honestly kind of slow and doesn't need rate limiting)
• Error handling for failed API requests
• Maintains row order even when some requests fail
• Proper CSV escaping for special characters
• Progress logging during processing

Requirements

• Node.js 14.0.0 or higher
• Valid Lob API key
• Input CSV files with id column containing postcard IDs