Node.js

WellSync, if you are gonna useSync this wonky language thing inSync one place, you might as well useSync it everywhereSync and make it more decent. See also: how to convert async to sync in JavaScript.

Their CLI debugger is a bit crap compared to GDB, basic functionality is either lacking or too verbose:

stackoverflow.com/questions/65493221/how-to-break-at-a-specific-function-or-line-with-the-node-js-node-inspect-comman
stackoverflow.com/questions/70486188/how-to-break-on-uncaught-exception-on-the-node-js-node-inspect-command-line-debu Some operations are only possible on the browser debug UI...

Documentation at: nodejs.org/dist/latest-v16.x/docs/api/debugger.html

Table of contents 3k 41

Node.js example

words: 723 articles: 3

Under nodejs:

nodejs/count.js

words: 12

This example counts to finity, sleeping 1 second between each count. Related:

stackoverflow.com/questions/14249506/how-can-i-wait-in-node-js-javascript-l-need-to-pause-for-a-period-of-time

nodejs/count.js

#!/usr/bin/env node
(async function() {
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))
let i = 0
let max
if (process.argv.length > 2) {
  max = parseInt(process.argv[2])
}
while (true) {
  console.log(i)
  if (i === max) break
  await sleep(1000)
  i++
}
})()

nodejs/read_child_process_lines.js

words: 24

This example reads lines from a child process one by one, as soon as lines become fully available. Related:

stackoverflow.com/questions/9781214/parse-output-of-spawned-node-js-child-process-line-by-line canon
- stackoverflow.com/questions/14332721/node-js-spawn-child-process-and-get-terminal-output-live/77423073#77423073 answered big superset
- stackoverflow.com/questions/12955154/nodejs-read-and-parse-each-line-of-stdout duplicate

nodejs/read_child_process_lines.js

#!/usr/bin/env node
const childProcess = require('child_process')
const readline = require('readline')
const p = childProcess.spawn('./count.js', ['2'])
;(async function() {
const rl = readline.createInterface({ input: p.stdout })
for await (const line of rl) {
  console.log(line)
}
})()

JavaScript memory usage benchmark

words: 683

In this section we will use the file nodejs/bench_mem.js, tests are run on Node.js v16.14.2 from NVM, Ubuntu 21.10, on Lenovo ThinkPad P51 (2017) which has 32 GB RAM.

First using topp from stackoverflow.com/questions/1221555/retrieve-cpu-usage-and-memory-usage-of-a-single-process-on-linux/40576129#40576129 let's observe the memory usage of some baseline cases.

A C hello world with an infinite loop at the end has:

2.7 MB
770 KB

For a Node.js infinite loop nodejs/infinite_loop.js

topp infinite_loop.js

This gives approximately:

RSS: 20 MB
VSZ: 230 MB

Adding a single hello world to it as in nodejs/infinite_hello.js and running:

topp infinite_hello.js

leads to:

RSS: 26 MB
VSZ: 580 MB

We understand that Node.js preallocates VSZ wildly. No big deal, but it does mean that VSZ is a useless measure for Node.js.

Forcing garbage collection as in nodejs/infinite_hello.js brings it down to 20 MB however:

topp node --expose-gc infinite_hello_gc.js

Finally let's see a baseline for process.memoryUsage nodejs/infinite_memoryusage.js:

node --expose-gc infinite_memoryusage.js

which gives initially:

{
  rss: 23851008,
  heapTotal: 6987776,
  heapUsed: 3674696,
  external: 285296,
  arrayBuffers: 10422
}

but after a few seconds randomly jumps to:

{
  rss: 26005504,
  heapTotal: 9084928,
  heapUsed: 3761240,
  external: 285296,
  arrayBuffers: 10422
}

so we understand that

heapUsed seems constant at 3.7 MB
heapTotal is a very noisy, as it starts at 7 MB, but randomly jumps to 9 MB at one point without apparent reason

Now let's run our main test program.

First a baseline case with an array of length 1:

node --expose-gc bench_mem.js n 1

This gives the same results as node --expose-gc infinite_memoryusage.js. The same result is obtained by doing:

a = undefined

with:

node --expose-gc bench_mem.js dealloc

Not let's vary the size of n a bit with:

node --expose-gc bench_mem.js n N

which gives:

N	`heapUsed`	`heapTotal`	`rss`	`heapUsed` per elem	`rss` per elem
1 M	14 MB	48 MB	56 MB	10	30
10 M	122 MB	157 MB	176 MB	18	15
100 M	906 MB	940 MB	960 MB	9	9.3

"rss per elem" is calculated as: rss - 26 MB, where 26 MB is the baseline RSS seen on n 1.

Similarly "heapUsed per elem" deduces the 4 MB (approximation of the above 3.7 MB) seen on n 1.

Note that to reach MAX_SAFE_INTEGER we would need 8 bytes per elem worst case.

Everything below 100 million (8) is therefore very memory wasteful in terms of RSS.

If we use Int32Array typed array buffers instead of a simple Array:

node --expose-gc bench_mem.js array-buffer n N

we see that the memory is now, unsurprisingly, accounted for under arrayBuffers, e.g. for N 1 million:

{
  rss: 31776768,
  heapTotal: 6463488,
  heapUsed: 3674520,
  external: 4285296,
  arrayBuffers: 4010422
}

Results for different N:

|| N
|| `arrayBuffers`
|| `rss`
|| `rss` per elem

| 1 M
| 4 MB
| 31 MB
| 5

| 10 M
| 40 MB
| 67 MB
| 4.6

| 100 M
| 40 MB
| 427 MB
| 4

We see therefore that typed arrays are much closer to what they advertise (4 bytes per element), even for smaller element counts, as expected.

Now let's try one million objects of type { a: 1, b: -1 }:

node --expose-gc bench_mem.js obj

gives:

{
  rss: 138969088,
  heapTotal: 105246720,
  heapUsed: 70103896,
  external: 285296,
  arrayBuffers: 10422
}

Disaster! Memory usage is up to 70 MB! Why?? We were expecting only about 24, 4 baseline + 2 * 10 for each million int?!

And now an equivalent version using class:

node --expose-gc bench_mem.js class

gives the same result.

Let's try Array:

node --expose-gc bench_mem.js arr

is even worse at 78 MB!! OMG why.

{
  rss: 164597760,
  heapTotal: 129363968,
  heapUsed: 78117008,
  external: 285296,
  arrayBuffers: 10422
}

Let's change the number of fields on the object? First as a sanity check:

node --expose-gc bench_mem.js obj 2

produces as expected the smae result as:

node --expose-gc bench_mem.js obj

so adding properties one by one doesn't change anything from creating the literal all at once. Good.

Now:

node --expose-gc bench_mem.js obj N

gives heapUsed:

1: 70M
2: 70M
3: 70M
4: 70M
5: 110M
6: 110M
7: 110M
8: 134M
9: 134M
10: 134M
11: 158M

Node.js library

words: 1k articles: 13

Node.js ORM library

words: 1k articles: 9

Sequelize

words: 194 articles: 8

This section is present in another page, follow this link to view it.

Node.js web framework

words: 1k articles: 19

Express.js

words: 187 articles: 4

This doesn't do a hole lot. Ciro Santilli wouldn't really call it a web framework. It's more like a middleware. Real web frameworks are built on top of it.

Examples under: nodejs/express:

nodejs/express/min.js: minimal example. Visit localhost:3000 and it shows hello world. It is a bit wrong because the headers say HTML but we return plaintext.
nodejs/express/index.js: example dump with automated tests where possible. The automated tests are run at startup after the server launches. Then the server keeps running so you can interact with it.

A live example on Heroku can be seen at: github.com/cirosantilli/heroku-node-min

Realworld app written in Express

words: 95 articles: 3

gothinkster/realworld implementations based on Express.js.

gothinkster/node-express-realworld-example-app

github.com/gothinkster/node-express-realworld-example-app

sigoden/node-express-realworld-example-app

words: 23

Appears to be a port of gothinkster/node-express-realworld-example-app to Sequelize.

Seemed to just work at 68bbadfd77f679f0df0fcd0de5bceb9c37b1144a Ubuntu 20.10, was forked from parent project in 2018.

Very raw. Easy to understand. Relatively well organiezd. But also very buggy at 3ab8d9f849a1cdf2985a8d123b1893f0fd4e79ab: github.com/Varun-Hegde/Conduit_NodeJS/issues/3, I just can't trust it. There must be several helper libraries that would greatly DRY up the repetitive CRUD. Ciro hates the style :-) 4 space indents, no space after commas, no semicolon. Not based on github.com/gothinkster/node-express-realworld-example-app which is essentially one of the reference implementations, so from scratch apparently, which is a bad sign.

FeathersJS

words: 718 articles: 8

Looks interesting.

It seems to abstract the part about the client messaging the backend, which focuses on being able to easily plug in a number of Front-end web framework to manage client state.

Has the "main web API is the same as the REST API" focus, which is fundamental 2020-nowadays.

Uses Socket.IO, which allows the client Javascript to register callbacks when data is updated to achieve Socket.IO, e.g. their default chat app does:

client.service('messages').on('created', addMessage);

so that message appear immediately as they are sent.

Their standard template from feathers generate app on @feathersjs/cli@4.5.0 includes:

several authentication methods, including OAuth
testing
backend database with one of several object-relational mapping! However, they don't abstract across them. E.g., the default Chat example uses NeDB, but a real app will likely use Sequelize, and a port is needed

which looks promising! They don't have a default template for a Front-end web framework however unfortunately: docs.feathersjs.com/guides/frameworks.html#the-feathers-chat lists a few chat app versions, which is their hello world:

Front-end web framework: not built-in on generator, but there are some sample repos pointed from the documentation, and they did work out-of-box:
- feathers-chat-react

But it is in itself a completely boring app with a single splash page, and no database interaction, so not a good showcase. The actual showcase app is feathersjs/feathers-chat.

And there is no official example of the chat app that is immediately deployable to Heroku: FeathersJS Heroku deployment, all setups require thinking.

Global source entry point: determine on package.json as usual, defaults to src/index.js.

FeatherJS demo apps

words: 444 articles: 5

github.com/feathersjs/awesome-feathersjs#projects-using-feathers

feathersjs/feathers-chat

words: 150 articles: 2

github.com/feathersjs/feathers-chat

The main FeathersJS hello world demo. Notable missing things...

instant Heroku deployability: FeathersJS Heroku deployment
no Front-end web framework which sucks, but there are basically official demos that worked e.g. feathers-chat-react
FeathersJS signup email verification

feathers-chat PostgreSQL

words: 19

The default feathers-chat app runs on NeDB (local filesystem JSON database).

Ciro Santilli managed to port it to Sequelize for PostgreSQL as shown at: github.com/cirosantilli/feathers-chat/tree/sequelize-pg

feathers-chat-react

words: 111

github.com/feathersjs-ecosystem/feathers-chat-react

Last updated 2018 as of 2021, but still just worked.

Also uses webpack which is fantastic.

Gotta run github.com/feathersjs/feathers-chat first: github.com/feathersjs-ecosystem/feathers-chat-react/issues/5, then it worked:

git clone https://github.com/feathersjs/feathers-chat
cd feathers-chat
git checkout fd729a47c57f9e6170cc1fa23cee0c84a004feb5
npm install
npm start

and on the other terminal:

git clone https://github.com/feathersjs-ecosystem/feathers-chat-react
cd feathers-chat-react
git checkout 36d56cbe80bbd5596f6a108b1de9db343b33dac3
npm install
npm start

then visit localhost:3000/ and you can create an account and login, tested on Ubuntu 20.10. Data is stored on persistently.

TODO how to merge those two repos into a single repo.

If you disable JavaScript on Chromium, it stops working completely. There is a section on how to solve that at: docs.feathersjs.com/cookbook/express/view-engine.html but it does not cover React specifically. Codaisseur/feathersjs-react-redux-ssr might be good to look into.

Codaisseur/feathersjs-react-redux-ssr

words: 243

github.com/Codaisseur/feathersjs-react-redux-ssr

Also webpack and Babel, looks promising!

As of 2021, last commit from 2017.

Running:

git clone https://github.com/Codaisseur/feathersjs-react-redux-ssr
cd feathersjs-react-redux-ssr
npm install

failed on Ubuntu 20.10 Node.js v14.15.3 with:

../src/create_string.cpp:17:37: error: no matching function for call to ‘v8::String::Utf8Value::Utf8Value(v8::Local<v8::Value>&)’
   17 |   v8::String::Utf8Value string(value);
      |                                     ^

Likely similar bullshit from: stackoverflow.com/questions/50111688/node-sqlite-node-gyp-build-error-no-member-named-forceset-in-v8object because the Node.js version is too new.

If I try nvm install v10

I Google error messages until reaching:

diff --git a/gulpfile.js b/gulpfile.js
index b931e06..24d2cc8 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -14,34 +14,34 @@ gulp.task('css', function() {
            .pipe(gulp.dest('./dist'))
 })
 
-gulp.task('css:watch', ['css'], function() {
+gulp.task('css:watch', gulp.series('css', function() {
   gulp.watch('app/styles/**/*.sass', ['css'])
-})
+}))
 
 gulp.task('moveAssets', function() {
   return gulp.src('./app/assets/**/*')
              .pipe(gulp.dest('./dist/assets'))
 })
 
-gulp.task('build:revAssets', ['css', 'moveAssets'], function() {
+gulp.task('build:revAssets', gulp.series('css', 'moveAssets', function() {
   var rev = new $.revAll()
   return gulp.src('./dist/**/*')
              .pipe(rev.revision())
              .pipe(gulp.dest('./dist/public'))
              .pipe(rev.manifestFile())
              .pipe(gulp.dest('./dist'))
-})
+}))
 
 gulp.task('build:cpServer', function() {
   return gulp.src('./app/**/*.{js,ejs}')
              .pipe(gulp.dest('./dist/server-build'))
 })
-gulp.task('build:revServer', ['build:cpServer'], function() {
+gulp.task('build:revServer', gulp.series('build:cpServer', function() {
   var manifest = gulp.src('./dist/rev-manifest.json')
   return gulp.src('./dist/server-build/{components,containers}/**/*')
              .pipe($.revReplace({ manifest: manifest }))
              .pipe(gulp.dest('./dist/server-build'))
-})
+}))
 
 gulp.task('build', function() {
   runSequence('build:revAssets', 'build:revServer')
diff --git a/package.json b/package.json
index bcb29c3..86bd593 100644
--- a/package.json
+++ b/package.json
@@ -67,7 +67,7 @@
     "redux-thunk": "^0.1.0",
     "request": "^2.79.0",
     "rewire": "^2.3.4",
-    "run-sequence": "^1.2.2",
+    "run-sequence": "^2.2.1",
     "serve-favicon": "^2.3.2",
     "socket.io-client": "^1.7.2",
     "superagent": "^1.4.0",
@@ -86,16 +86,16 @@
     "concurrently": "^2.0.0",
     "cross-env": "^1.0.7",
     "enzyme": "^2.3.0",
-    "gulp": "^3.9.0",
+    "gulp": "^4.0.2",
     "gulp-autoprefixer": "^3.1.0",
     "gulp-load-plugins": "^1.2.0",
     "gulp-rev": "^6.0.1",
-    "gulp-sass": "^2.1.1",
+    "gulp-sass": "4.1.0",
     "gulp-sourcemaps": "^1.6.0",
     "jsdom": "^7.0.1",
     "mocha": "^2.4.5",
     "nock": "^2.17.0",
-    "node-sass": "^3.4.2",
+    "node-sass": "^5.0.0",
     "nodemon": "^1.6.0",
     "react-addons-test-utils": "^15.3.2",
     "react-transform-catch-errors": "^1.0.0",

and the next problem is: stackoverflow.com/questions/48513573/gulp-error-gulp-hastask-is-not-a-function

randyscotsmithey/feathers-realworld-example-app

words: 51

FeathersJS entry for gothinkster/realworld.

MongoDB-based.

So once you install MongoDB, run with:

MONGODB_FEATHERS_REALWORLD=mongodb://localhost:27017/mydb npm start

Got it working on Ubuntu 20.10 with both React and Vue.js front-ends at github.com/randyscotsmithey/feathers-realworld-example-app/commit/8bc3a09242285de624c75bb8345630df499a7d07 as mentioned at github.com/randyscotsmithey/feathers-realworld-example-app/issues/2 except for bad error reporting on UI.

Tests can be run with:

MONGODB_FEATHERS_REALWORLD=mongodb://localhost:27017/mydb npm run test

but there were 10 failures and 55 passes: github.com/randyscotsmithey/feathers-realworld-example-app/issues/3

FeathersJS Heroku deployment

words: 42

Got it working as mentioned at: github.com/cirosantilli/feathers-chat/tree/sequelize-pg

One major step was to port to PostgreSQL as shown at feathers-chat PostgreSQL.

Bibliography:

There's also a heroku branch at: github.com/feathersjs/feathers-chat/tree/heroku, but it also seems to use NeDB? So you can have a filesystem in Heroku? Doesn't seem so: stackoverflow.com/questions/42775418/heroku-local-persistent-storage

Meteor (web framework)

words: 33

github.com/meteor/meteor

The idea is cool. It really unifies front-and back end.

But Ciro Santilli feels the approach proposed by FeathersJS of being a glue between bigger third-party Front-end web frameworks like React and backend (object-relational mapping) is more promising and flexible.

Nest.js

words: 157 articles: 2

lujakob/nestjs-realworld-example-app

words: 153 articles: 1

Nest.js entry for gothinkster/realworld.

Didn't manage to get it to work perfectly on Ubuntu 20.10: github.com/lujakob/nestjs-realworld-example-app/issues/57

lujakob/nestjs-realworld-example-app SQLite port

words: 140

Tried a quick port to SQLite to get rid of annoying local databases for development, but failed, at c1c2cc4e448b279ff083272df1ac50d20c3304fa

npm install sqlite3 --save-dev

and

{
  "type": "sqlite",
  "database": "db.sqlite3",
  "entities": ["src/**/**.entity{.ts,.js}"],
  "synchronize": true
}

then:

npm start

fails with:

DataTypeNotSupportedError: Data type "timestamp" in "ArticleEntity.created" is not supported by "sqlite" database.

Attempt to hack it:

--- a/src/article/article.entity.ts
+++ b/src/article/article.entity.ts
@@ -20,10 +20,10 @@ export class ArticleEntity {
   @Column({default: ''})
   body: string;

-  @Column({ type: 'timestamp', default: () => "CURRENT_TIMESTAMP"})
+  @Column({ default: () => "CURRENT_TIMESTAMP"})
   created: Date;

-  @Column({ type: 'timestamp', default: () => "CURRENT_TIMESTAMP"})
+  @Column({ default: () => "CURRENT_TIMESTAMP"})
   updated: Date;

and after that it seems to run.

I can signup and login, terrible error reporting as usual, make sure to use long enough usernames/passwords.

However, article creation fails with:

Unhandled Rejection (TypeError): Cannot read property 'slug' of undefined

Sails.js

words: 33

github.com/balderdashy/sails

Front-end web framework integration: no native one:

React:
- github.com/markmur/sails-react-webpack
- github.com/sepineda/sails-react-webpack Last updated as of 2021: 2018
- stackoverflow.com/questions/28331032/sails-js-with-react-js-how-to-do-it-correctly
Vue.js:
- github.com/mikermcneil/ration Issue tracker disabled...
  - live at: ration.io/
  - selling a course at: courses.platzi.com/courses/sails-js/
- platzi.com/cursos/javascript-pro/ non-free and in Spanish pointed to from official README...
- Nuxt.js:
  - github.com/AngelMunoz/sails-nuxt

TODO server-side rendering anyone??

NVM

words: 6

github.com/nvm-sh/nvm

The best way to install Node.js: