Add --verbose flag for HTTP debugging and clarify Bluesky identifier format by Copilot · Pull Request #146 · humanwhocodes/crosspost

Copilot · 2025-10-13T18:48:27Z

Problem

Users attempting to debug network issues with various social media integrations (particularly Bluesky) were encountering opaque error messages that made troubleshooting difficult. For example:

❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Without visibility into the actual HTTP responses, users couldn't determine whether the issue was due to incorrect credentials, wrong identifier format, server errors, or other problems.

Additionally, the README documentation for Bluesky setup didn't clearly specify whether the identifier should include the @ symbol or use the DID format, leading to configuration confusion.

Solution

1. Added `--verbose` CLI Flag

Implemented comprehensive HTTP request/response logging that activates when the --verbose flag is present. The logging system:

Captures all HTTP requests: URL, method, headers, and request body
Captures all HTTP responses: Status code, headers, and response body (regardless of success or failure)
Handles multiple content types: JSON responses are pretty-printed, text responses are displayed as-is
Includes security safeguards:
- Redacts sensitive headers (Authorization, Cookie, Set-Cookie, X-API-Key, API-Key)
- Redacts sensitive request body fields (password, access_token, api_key)
- Truncates large payloads to 5000 characters to prevent memory exhaustion and console overflow

Example output:

$ crosspost --verbose --bluesky "Testing"

--- HTTP Request ---
URL: https://bsky.social/xrpc/com.atproto.server.createSession
Method: POST
Headers: {
  "Content-Type": "application/json"
}
Body: {"identifier":"username.bsky.social","password":"[REDACTED]"}

--- HTTP Response ---
Status: 401 Unauthorized
Headers: {
  "content-type": "text/html"
}
Body: <!DOCTYPE html>
<html>
<head><title>401 Unauthorized</title></head>
<body><h1>401 Unauthorized</h1></body>
</html>
--- End Response ---

❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Now users can immediately see that the server returned an HTML error page instead of JSON, indicating an authentication problem.

2. Updated Bluesky README Documentation

Clarified the BLUESKY_IDENTIFIER format in the setup section:

Should be the human-readable username without the @ symbol (e.g., username.bsky.social)
Should not be the DID (Decentralized Identifier like did:plc:...)

Implementation Details

The verbose logging uses a global fetch wrapper approach, which is minimal and non-invasive:

Only wraps fetch when --verbose flag is present (zero performance impact otherwise)
No changes required to existing strategy implementations
Properly handles different header object types (Headers API vs plain objects)
Robust error handling for various request/response body types

Testing

All 299 existing tests continue to pass
Added new test to verify --verbose flag appears in help output
Linting and type checking pass
Full test suite (unit, build, type tests) passes

Fixes #[issue-number]

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

httpbin.org
- Triggering command: node /tmp/test-verbose.js --verbose (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

This section details on the original issue you should resolve

<issue_title>Debug option for responses from networks</issue_title>
<issue_description>I am currently setting up various strategies, but I'm having trouble getting my Bluesky integration to work. It would be nice if crosspost --verbose or some other option would be available that might print a readable output of the response from the network:
> crosspost --bluesky "Testing"
❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at parseJSONFromBytes (node:internal/deps/undici/undici:6223:19)
    at successSteps (node:internal/deps/undici/undici:6204:27)
    at readAllBytes (node:internal/deps/undici/undici:5173:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
The above is what I get when I call. I assume an error page is returned, but I don't know its content at this point. Is there an easy way to somehow log (either into the terminal or a logfile) what the responses are?

Regarding the Bluesky issue, I assume it's the IDENTIFIER for my user. The docs are not very clear about what identifier that is (Username with or without @, DID from Bluesky). We cross that bridge when we come to it :)</issue_description>

<agent_instructions>Add a --verbose CLI option that, when present, causes the CLI to output the HTTP response from each request to the console. This should happen regardless of the HTTP status code.

We also need to update the README for the BlueSky strategy to indicate that the username must be the username without @.</agent_instructions>

Comments on the Issue (you are @copilot in this section)
@nzakas Sorry I missed this. I've been buried under notifications for a while now.
Yes, I think we can add a --verbose flag that outputs the raw response from each request. I think it should do so regardless of response success or failure to keep with the meaning of the flag even if it's most helpful for failures.

And we can update the README so that it states the Bluesky username should not contain @ and should be the human-readable username and not the DiD</comment_new>

Fixes #142

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

…truncation Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

… improve header handling Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

nzakas · 2025-10-13T20:30:30Z

README.md

-**Important:** Do not use your login password with the API.
+**Important:** 
+- Do not use your login password with the API.
+- The identifier should be your human-readable username without the `@` symbol (e.g., `username.bsky.social`), not your DID (Decentralized Identifier).


Suggested change

- The identifier should be your human-readable username without the `@` symbol (e.g., `username.bsky.social`), not your DID (Decentralized Identifier).

- The identifier should be your human-readable username without the `@` symbol and without the host (e.g., `username`), not your DID (Decentralized Identifier).

Initial plan

2d04df0

Copilot AI assigned Copilot and nzakas Oct 13, 2025

Copilot started work on behalf of nzakas October 13, 2025 18:49 View session

Copilot AI and others added 4 commits October 13, 2025 18:52

Initial plan for verbose flag and Bluesky README update

ff33302

Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

Add --verbose flag and update Bluesky README

2822ae7

Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

Improve verbose logging: add sensitive header filtering and response …

144ab5e

…truncation Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

Enhance verbose logging: redact password fields in request bodies and…

c9c872b

… improve header handling Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add debug option for network response output~~ Add --verbose flag for HTTP debugging and clarify Bluesky identifier format Oct 13, 2025

Copilot AI requested a review from nzakas October 13, 2025 19:05

Copilot finished work on behalf of nzakas October 13, 2025 19:05

nzakas reviewed Oct 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add --verbose flag for HTTP debugging and clarify Bluesky identifier format#146

Add --verbose flag for HTTP debugging and clarify Bluesky identifier format#146
Copilot wants to merge 5 commits intomainfrom
copilot/add-debug-option-for-network-responses

Copilot AI commented Oct 13, 2025 •

edited

Loading

Uh oh!

nzakas Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	- The identifier should be your human-readable username without the `@` symbol (e.g., `username.bsky.social`), not your DID (Decentralized Identifier).
	- The identifier should be your human-readable username without the `@` symbol and without the host (e.g., `username`), not your DID (Decentralized Identifier).

Uh oh!

Conversation

Copilot AI commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

1. Added --verbose CLI Flag

2. Updated Bluesky README Documentation

Implementation Details

Testing

I tried to connect to the following addresses, but was blocked by firewall rules:

Comments on the Issue (you are @copilot in this section)

Uh oh!

nzakas Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Oct 13, 2025 •

edited

Loading

1. Added `--verbose` CLI Flag