node-red-contrib-comment-scraper 1.0.0

Node-RED node that reads Reddit and Hacker News threads exported with the Comment Scraper browser extension (JSON or CSV) and turns them into messages.

npm install node-red-contrib-comment-scraper

node-red-contrib-comment-scraper

A Node-RED node that reads Reddit and Hacker News threads exported with the Comment Scraper browser extension. It turns an export file into messages: one per thread, or one per comment.

The node doesn't go to Reddit or Hacker News. You export the thread in your browser, then read the file in a flow. Everything after that stays local: no network requests and no dependencies.

How to install the extension and export a thread is covered in the Comment Scraper docs.

Install

In Node-RED, open Manage palette, go to Install and search for node-red-contrib-comment-scraper. Or run this in your Node-RED user directory (usually ~/.node-red) and restart Node-RED:

npm install node-red-contrib-comment-scraper

Needs Node-RED 3.0 or newer and Node.js 18 or newer.

Usage

The node takes the contents of an export file in msg.payload. The usual flow is:

[inject] -> [file in] -> [comment scraper] -> [debug]

Set the file in node to the path of the export and its output to "a single utf8 string" (a single Buffer works too). If a json node has already parsed the file, the node takes that object as well.

It reads JSON exports, flat or nested, and CSV exports, including batch exports with several threads in one file. Markdown and Org exports can't be read.

To pick up new exports on their own, put a watch node on the folder your browser saves downloads to, add a switch node that only lets through msg.filename ending in .json or .csv, and set the file in node's filename to msg.filename.

Options

• Send: "one message per thread" (the default) or "one message per comment".
• Leave out deleted and removed comments: off by default. The remaining comments keep their numbers.

Output

One message per thread

msg.payload is the thread. Trimmed:

{
  "source": "reddit",
  "thread_id": "pm1",
  "thread_title": "What made you switch project management tools?",
  "subreddit": "smallbusiness",
  "permalink": "https://www.reddit.com/r/smallbusiness/comments/pm1/",
  "fetched_at": "2026-10-10T12:00:00.000Z",
  "task_state": "all_known_fetched",
  "fetch_path": "json",
  "post": {
    "record_type": "post",
    "body": "Our team of six is about to switch. What pushed you to move?",
    "score": 50,
    "is_op": true
  },
  "comments": [
    {
      "comment_id": "p1",
      "parent_id": "pm1",
      "depth": 0,
      "score": 42,
      "is_op": false,
      "comment_state": "normal",
      "body": "Per-seat pricing. We went from 6 to 14 people and the bill tripled.",
      "permalink": "https://www.reddit.com/r/smallbusiness/comments/pm1/_/p1/",
      "parent_missing": false,
      "comment_number": 1,
      "reply_to_number": null
    },
    {
      "comment_id": "p2",
      "parent_id": "p1",
      "depth": 1,
      "score": 12,
      "is_op": true,
      "comment_state": "normal",
      "body": "That's what worries us. Did you move everything at once?",
      "permalink": "https://www.reddit.com/r/smallbusiness/comments/pm1/_/p2/",
      "parent_missing": false,
      "comment_number": 2,
      "reply_to_number": 1
    }
  ]
}

JSON exports also carry stop_reason, fetched, known_unfetched_at_least and reported_by_source on the thread. CSV exports don't have those.

One message per comment

msg.payload is a single comment, msg.thread is the thread with its post but without comments, and msg.parts lets a join node collect the comments of each thread back into an array.

Fields

Posts and comments keep the field names of the export: comment_id, parent_id, depth, score, score_hidden, created_utc, edited_utc, is_op, comment_state, body, permalink and the rest. The export formats section explains each one. A few things worth knowing:

• Comments come in reading order: each comment, then its replies. depth is 0 for a top-level comment.
• comment_number is the number the extension's Markdown export shows as [c12]. reply_to_number is the number of the comment being answered, or null for top-level comments.
• A reply whose parent isn't in the export has parent_missing: true and reply_to_number: null.
• Exports have no usernames. is_op is true when the comment came from the person who started the thread.
• Deleted and removed comments keep their place with an empty body and comment_state set to "deleted" or "removed", so the replies under them still have a parent.
• Other properties of the incoming message, such as msg.filename and msg.topic, are passed on.

CSV and formulas

The extension puts a ' in front of CSV cells that start with =, +, - or @ so spreadsheets don't run them as formulas. The node removes it again. A comment that really started with '- comes back as -, so use JSON when you need every character exactly.

Errors

If msg.payload isn't a Comment Scraper export, or the file looks cut off or hand-edited (a missing field, a short CSV row, a repeated comment ID), the node sends nothing, shows a red status and passes the error to any catch node. The error says what went wrong and, where it can, which thread or CSV row it found the problem in.

Examples

In Node-RED, go to Import > Examples > node-red-contrib-comment-scraper:

• Read a thread export holds a short sample thread and runs as is. It shows both output modes side by side.
• Read an export file reads a file from disk. Set the path in the file in node first.

Tests

npm install
npm test

The test exports in test/fixtures were written by the extension's own export code (see test/fixtures/generate.ts).

License

MIT