diff --git a/docs/npm_nodejs.md b/docs/npm_nodejs.md new file mode 100644 index 0000000000..b2f0d438c9 --- /dev/null +++ b/docs/npm_nodejs.md @@ -0,0 +1,59 @@ +# Using npm/Node.js code + +While Deno is pretty powerful itself, there is a large eco-system of code in the +[npm](https://npmjs.com/) registry, and many people will want to re-leverage +tools, code and libraries that are built for [Node.js](https://nodejs.org/). In +this chapter we will explore how to use it. + +The good news, is that in many cases, it _just works_. + +There are some foundational things to understand about differences between +Node.js and Deno that can help in understanding what challenges might be faced: + +- Current Node.js supports both CommonJS and ES Modules, while Deno only + supports ES Modules. The addition of stabilized ES Modules in Node.js is + relatively recent and most code written for Node.js is in the CommonJS format. +- Node.js has quite a few built-in modules that can be imported and they are a + fairly expansive set of APIs. On the other hand, Deno focuses on implementing + web standards and where functionality goes beyond the browser, we locate APIs + in a single global `Deno` variable/namespace. Lots of code written for Node.js + expects/depends upon these built-in APIs to be available. +- Node.js has a non-standards based module resolution algorithm, where you can + import bare-specifiers (e.g. `react` or `lodash`) and Node.js will look in + your local and global `node_modules` for a path, introspect the `package.json` + and try to see if there is a module named the right way. Deno resolves modules + the same way a browser does. For local files, Deno expects a full module name, + including the extension. When dealing with remote imports, Deno expects the + web server to do any "resolving" and provide back the media type of the code + (see the + [Determining the type of file](../typescript/overview.md#determining-the-type-of-file) + for more information). +- Node.js effectively doesn't work without a `package.json` file. Deno doesn't + require an external meta-data file to function or resolve modules. +- Node.js doesn't support remote HTTP imports. It expects all 3rd party code to + be installed locally on your machine using a package manager like `npm` into + the local or global `node_modules` folder. Deno supports remote HTTP imports + (as well as `data` and `blob` URLs) and will go ahead and fetch the remote + code and cache it locally, similar to the way a browser works. + +In order to help mitigate these differences, we will further explore in this +chapter: + +- Using the [`std/node`](./npm_nodejs/std_node.md) modules of the Deno standard + library to "polyfill" the built-in modules of Node.js +- Using [CDNs](./npm_nodejs/cdns.md) to access the vast majority of npm packages + in ways that work under Deno. +- How [import maps](./npm_nodejs/import_maps.md) can be used to provide "bare + specifier" imports like Node.js under Deno, without needing to use a package + manager to install packages locally. +- And finally, a general section of [frequently asked + questions][./npm_nodejs/faqs.md] + +That being said, there are some differences that cannot be overcome: + +- Node.js has a plugin system that is incompatible with Deno, and Deno will + never support Node.js plugins. If the Node.js code you want to use requires a + "native" Node.js plugin, it won't work under Deno. +- Node.js has some built in modules (e.g. like `vm`) that are effectively + incompatible with the scope of Deno and therefore there aren't easy ways to + provide a _polyfill_ of the functionality in Deno. diff --git a/docs/npm_nodejs/cdns.md b/docs/npm_nodejs/cdns.md new file mode 100644 index 0000000000..36b26199fc --- /dev/null +++ b/docs/npm_nodejs/cdns.md @@ -0,0 +1,186 @@ +## Packages from CDNs + +Because Deno supports remote HTTP modules, and content delivery networks (CDNs) +can be powerful tools to transform code, the combination allows an easy way to +access code in the npm registry via Deno, usually in a way that works with Deno +without any further actions, and often enriched with TypeScript types. In this +section we will explore that in detail. + +### What about `deno.land/x/`? + +The [`deno.land/x/`](https://deno.land/x/) is a public registry for code, +hopefully code written specifically for Deno. It is a public registry though and +all it does is "redirect" Deno to the location where the code exists. It doesn't +transform the code in any way. There is a lot of great code on the registry, but +at the same time, there is some code that just isn't well maintained (or doesn't +work at all). If you are familiar with the npm registry, you know that as well, +there are varying degrees of quality. + +Because it simply serves up the original published source code, it doesn't +really help when trying to use code that didn't specifically consider Deno when +authored. + +### Deno "friendly" CDNs + +Deno friendly content delivery networks (CDNs) not only host packages from npm, +they provide them in a way that maximizes their integration to Deno. They +directly address some of the challenges in consuming code written for Node.js: + +- The provide packages and modules in the ES Module format, irrespective of how + they are published on npm. +- They resolve all the dependencies as the modules are served, meaning that all + the Node.js specific module resolution logic is handled by the CDN. +- Often, they inform Deno of type definitions for a package, meaning that Deno + can use them to type check your code and provide a better development + experience. +- The CDNs also "polyfill" the built-in Node.js modules, making a lot of code + that leverages the built-in Node.js modules _just work_. +- The CDNs deal with all the semver matching for packages that a package manager + like `npm` would be required for a Node.js application, meaning you as a + developer can express your 3rd party dependency versioning as part of the URL + you use to import the package. + +#### esm.sh + +[esm.sh](https://esm.sh/) is a CDN that was specifically designed for Deno, +though addressing the concerns for Deno also makes it a general purpose CDN for +accessing npm packages as ES Module bundles. esm.sh uses +[esbuild](https://esbuild.github.io/) to take an arbitrary npm package and +ensure that it is consumable as an ES Module. In many cases you can just import +the npm package into your Deno application: + +```ts +import React from "https://esm.sh/react"; + +export default class A extends React.Component { + render() { + return ( +
+ ); + } +} +``` + +esm.sh supports the use of both specific versions of packages, as well as +[semver](https://semver.npmjs.com/) versions of packages, so you can express +your dependency in a similar way you would in a `package.json` file when you +import it. For example, to get a specific version of a package: + +```ts +import React from "https://esm.sh/react@17.0.2"; +``` + +Or to get the latest patch release of a minor release: + +```ts +import React from "https://esm.sh/react@~16.13.0"; +``` + +esm.sh uses the `std/node` polyfills to replace the built-in modules in Node.js, +meaning that code that uses those built-in modules will have the same +limitations and caveats as those modules in `std/node`. + +esm.sh also automatically sets a header which Deno recognizes that allows Deno +to be able to retrieve type definitions for the package/module. See +[Using `X-TypeScript-Types` header](../typescript/types.md#using-x-typescript-types-header) +in this manual for more details on how this works. + +The CDN is also a good choice for people who develop in mainland China, as the +hosting of the CDN is specifically designed to work with "the great firewall of +China", as well as esm.sh provides information on self hosting the CDN as well. + +Check out the [esm.sh homepage](https://esm.sh/) for more detailed information +on how the CDN can be used and what features it has. + +#### Skypack + +[Skypack.dev](https://www.skypack.dev/) is designed to make development overall +easier by not requiring packages to be installed locally, even for Node.js +development, and to make it easy to create web and Deno applications that +leverage code from the npm registry. + +Skypack has a great way of discovering packages in the npm registry, by +providing a lot of contextual information about the package, as well as a +"scoring" system to try to help determine if the package follows best-practices. + +Skypack detects Deno's user agent when requests for modules are received and +ensures the code served up is tailored to meet the needs of Deno. The easiest +way to load a package is to use the +[lookup URL](https://docs.skypack.dev/skypack-cdn/api-reference/lookup-urls) for +the package: + +```ts +import React from "https://cdn.skypack.dev/react"; + +export default class A extends React.Component { + render() { + return ( +
+ ); + } +} +``` + +Lookup URLs can also contain the [semver](https://semver.npmjs.com/) version in +the URL: + +```ts +import React from "https://cdn.skypack.dev/react@~16.13.0"; +``` + +By default, Skypack does not set the types header on packages. In order to have +the types header set, which is automatically recognized by Deno, you have to +append `?dts` to the URL for that package: + +```ts +import { pathToRegexp } from "https://cdn.skypack.dev/path-to-regexp?dts"; + +const re = pathToRegexp("/path/:id"); +``` + +See +[Using `X-TypeScript-Types` header](../typescript/types.md#using-x-typescript-types-header) +in this manual for more details on how this works. + +Skypack docs have a +[specific page on usage with Deno](https://docs.skypack.dev/skypack-cdn/code/deno) +for more information. + +### Other CDNs + +There are a couple of other CDNs worth mentioning. + +#### UNPKG + +[UNPKG](https://unpkg.com/) is the most well known CDN for npm packages. For +packages that include an ES Module distribution for things like the browsers, +many of them can be used directly off of UNPKG. That being said, everything +available on UNPKG is available on more Deno friendly CDNs. + +#### JSPM + +The [jspm.io](https://jspm.io) CDN is specifically designed to provide npm and +other registry packages as ES Modules in a way that works well with import maps. +While it doesn't currently cater to Deno, the fact that Deno can utilize import +maps, allows you to use the [JSPM.io generator](https://generator.jspm.io/) to +generate an import-map of all the packages you want to use and have them served +up from the CDN. + +### Considerations + +While CDNs can make it easy to allow Deno to consume packages and modules from +the npm registry, there can still be some things to consider: + +- Deno does not (and will not) support Node.js plugins. If the package requires + a native plugin, it won't work under Deno. +- Dependency management can always be a bit of a challenge and a CDN can make it + a bit more obfuscated what dependencies are there. You can always use + `deno info` with the module or URL to get a full breakdown of how Deno + resolves all the code. +- While the Deno friendly CDNs try their best to serve up types with the code + for consumption with Deno, lots of types for packages conflict with other + packages and/or don't consider Deno, which means you can often get strange + diagnostic message when type checking code imported from these CDNs, though + skipping type checking will result in the code working perfectly fine. This is + a fairly complex topic and is covered in the + [Types and type declarations](../typescript/types.md) section of the manual. diff --git a/docs/npm_nodejs/faqs.md b/docs/npm_nodejs/faqs.md new file mode 100644 index 0000000000..cde1c593a1 --- /dev/null +++ b/docs/npm_nodejs/faqs.md @@ -0,0 +1,55 @@ +## Frequently asked questions + +### Getting errors when type checking like `cannot find namespace NodeJS` + +One of the modules you are using has type definitions that depend upon the +NodeJS global namespace, but those types don't include the NodeJS global +namespace in their types. + +The quickest fix is to skip type checking. You can do this by using the +`--no-check` flag. + +Skipping type checking might not be acceptable though. You could try to load the +Node.js types yourself. For example from UNPKG it would look something like +this: + +```ts +import type {} from "https://unpkg.com/@types/node/index.d.ts"; +``` + +Or from esm.sh: + +```ts +import type {} from "https://esm.sh/@types/node/index.d.ts"; +``` + +Or from Skypack: + +```ts +import type {} from "https://cdn.skypack.dev/@types/node/index.d.ts"; +``` + +You could also try to provide only specifically what the 3rd party package is +missing. For example the package `@aws-sdk/client-dynamodb` has a dependency on +the `NodeJS.ProcessEnv` type in its type definitions. In one of the modules of +your project that imports it as a dependency, you could put something like this +in there which will solve the problem: + +```ts +declare global { + namespace NodeJS { + type ProcessEnv = Record; + } +} +``` + +### Getting type errors like cannot find `document` or `HTMLElement` + +The library you are using has dependencies on the DOM. This is common for +packages that are designed to run in a browser as well as server-side. By +default, Deno only includes the libraries that are directly supported. Assuming +the package properly identifies what environment it is running in at runtime it +is "safe" to use the DOM libraries to type check the code. For more information +on this, check out the +[Targeting Deno and the Browser](../typescript/configuration.md#targeting-deno-and-the-browser) +section of the manual. diff --git a/docs/npm_nodejs/import_maps.md b/docs/npm_nodejs/import_maps.md new file mode 100644 index 0000000000..ec78d6c115 --- /dev/null +++ b/docs/npm_nodejs/import_maps.md @@ -0,0 +1,112 @@ +## Using import maps + +Deno supports [import maps](../linking_to_external_code/import_maps.md) which +allow you to supply Deno with information about how to resolve modules that +overrides the default behavior. Import maps are a web platform standard that is +increasingly being included natively in browsers. They are specifically useful +with adapting Node.js code to work well with Deno, as you can use import maps to +map "bare" specifiers to a specific module. + +When coupled with Deno friendly [CDNs](./cdns.md) import maps can be a powerful +tool in managing code and dependencies without need of a package management +tool. + +### Bare and extension-less specifiers + +Deno will only load a fully qualified module, including the extension. The +import specifier needs to either be relative or absolute. Specifiers that are +neither relative or absolute are often called "bare" specifiers. For example +`"./lodash/index.js"` is a relative specifier and +`https://cdn.skypack.dev/lodash` is an absolute specifier. Where is `"lodash"` +would be a bare specifier. + +Also Deno requires that for local modules, the module to load is fully +resolve-able. When an extension is not present, Deno would have to "guess" what +the author intended to be loaded. For example does `"./lodash"` mean +`./lodash.js`, `./lodash.ts`, `./lodash.tsx`, `./lodash.jsx`, +`./lodash/index.js`, `./lodash/index.ts`, `./lodash/index.jsx`, or +`./lodash/index.tsx`? + +When dealing with remote modules, Deno allows the CDN/web server define whatever +semantics around resolution the server wants to define. It just treats a URL, +including its query string, as a "unique" module that can be loaded. It expects +the CDN/web server to provide it with a valid media/content type to instruct +Deno how to handle the file. More information on how media types impact how Deno +handles modules can be found in the +[Determining the type of file](../typescript/overview.md#determining-the-type-of-file) +section of the manual. + +Node.js does have defined semantics for resolving specifiers, but they are +complex, assume unfettered access to the local file system to query it. Deno has +chosen not to go down that path. + +But, import maps can be used to provide some of the ease of the developer +experience if you wish to use bare specifiers. For example, if we want to do the +following in our code: + +```ts +import lodash from "lodash"; +``` + +We can accomplish this using an import map, and we don't even have to install +the `lodash` package locally. We would want to create a JSON file (for example +**import_map.json**) with the following: + +```json +{ + "imports": { + "lodash": "https://cdn.skypack.dev/lodash" + } +} +``` + +And we would run our program like: + +``` +> deno run --import-map ./import_map.json example.ts +``` + +If you wanted to manage the versions in the import map, you could do this as +well. For example if you were using Skypack CDN, you can used a +[pinned URL](https://docs.skypack.dev/skypack-cdn/api-reference/pinned-urls-optimized) +for the dependency in your import map. For example, to pin to lodash version +4.17.21 (and minified production ready version), you would do this: + +```json +{ + "imports": { + "lodash": "https://cdn.skypack.dev/pin/lodash@v4.17.21-K6GEbP02mWFnLA45zAmi/mode=imports,min/optimized/lodash.js" + } +} +``` + +### Overriding imports + +The other situation where import maps can be very useful is the situation where +you have tried your best to make something work, but have failed. For example +you are using an npm package which has a dependency on some code that just +doesn't work under Deno, and you want to substitute another module that +"polyfills" the incompatible APIs. + +For example, let's say we have a package that is using a version of the built in +`"fs"` module that we have a local module we want to replace it with when it +tries to import it, but we want other code we are loading to use the standard +library replacement module for `"fs"`. We would want to create an import map +that looked something like this: + +```ts +{ + "imports": { + "fs": "https://deno.land/std@$STD_VERSION/node/fs.ts" + }, + "scopes": { + "https://deno.land/x/example": { + "fs": "./patched/fs.ts" + } + } +} +``` + +Import maps can be very powerful, check out the official +[standards README](https://github.com/WICG/import-maps#the-import-map) for more +information. diff --git a/docs/npm_nodejs/std_node.md b/docs/npm_nodejs/std_node.md new file mode 100644 index 0000000000..38327121c9 --- /dev/null +++ b/docs/npm_nodejs/std_node.md @@ -0,0 +1,100 @@ +## The `std/node` library + +The `std/node` part of the Deno standard library is a Node.js compatibility +layer. It's primary focus is providing "polyfills" for Node.js's +[built-in modules](https://github.com/denoland/deno_std/tree/main/node#supported-builtins). +It also provides a mechanism for loading CommonJS modules into Deno. + +The library is most useful when trying to use your own or private code that was +written for Node.js. If you are trying to consume public npm packages, you are +likely to get a better result using a [CDN](./cdns.md). + +### Node.js built-in modules + +The standard library provides several "replacement" modules for Node.js built-in +modules. These either replicate the functionality of the built-in or they call +the Deno native APIs, returning an interface that is compatible with Node.js. + +The standard library provides modules for the the following built-ins: + +- [`assert`](https://doc.deno.land/https/deno.land/std/node/assert.ts) + (_partly_) +- [`buffer`](https://doc.deno.land/https/deno.land/std/node/buffer.ts) +- [`child_process`](https://doc.deno.land/https/deno.land/std/node/child_process.ts) + (_partly_) +- [`console`](https://doc.deno.land/https/deno.land/std/node/console.ts) + (_partly_) +- [`constants`](https://doc.deno.land/https/deno.land/std/node/constants.ts) + (_partly_) +- [`crypto`](https://doc.deno.land/https/deno.land/std/node/crypto.ts) + (_partly_) +- [`events`](https://doc.deno.land/https/deno.land/std/node/events.ts) +- [`fs`](https://doc.deno.land/https/deno.land/std/node/fs.ts) (_partly_) +- [`module`](https://doc.deno.land/https/deno.land/std/node/module.ts) +- [`os`](https://doc.deno.land/https/deno.land/std/node/os.ts) (_partly_) +- [`path`](https://doc.deno.land/https/deno.land/std/node/path.ts) +- [`process`](https://doc.deno.land/https/deno.land/std/node/process.ts) + (_partly_) +- [`querystring`](https://doc.deno.land/https/deno.land/std/node/querystring.ts) +- [`stream`](https://doc.deno.land/https/deno.land/std/node/stream.ts) +- [`string_decoder`](https://doc.deno.land/https/deno.land/std/node/string_decoder.ts) +- [`timers`](https://doc.deno.land/https/deno.land/std/node/timers.ts) +- [`tty`](https://doc.deno.land/https/deno.land/std/node/tty.ts) (_partly_) +- [`url`](https://doc.deno.land/https/deno.land/std/node/url.ts) +- [`util`](https://doc.deno.land/https/deno.land/std/node/util.ts) (_partly_) + +In addition, there is the +[`std/node/global.ts`](https://doc.deno.land/https/deno.land/std/node/global.ts) +module which provides some of the Node.js globals like `global`, `process`, and +`Buffer`. + +If you want documentation for any of the modules, you can simply type `deno doc` +and the URL of the module in your terminal: + +``` +> deno doc https://deno.land/std/assert.ts +``` + +### Loading CommonJS modules + +Deno only supports natively loading ES Modules, but a lot of Node.js code is +still written in the CommonJS format. As mentioned above, for public npm +packages, it is often better to use a CDN to transpile CommonJS modules to ES +Modules for consumption by Deno. Not only do you get reliable conversion plus +all the dependency resolution and management without requiring a package manager +to install the packages on your local machine, you get the advantages of being +able to share your code easier without requiring other users to setup some of +the Node.js infrastructure to consume your code with Node.js. + +That being said, the built-in Node.js module `"module"` provides a function +named `createRequire()` which allows you to create a Node.js compatible +`require()` function which can be used to load CommonJS modules, as well as use +the same resolution logic that Node.js uses when trying to load a module. +`createRequire()` will also install the Node.js globals for you. + +Example usage would look like this: + +```ts +import { createRequire } from "https://deno.land/std@$STD_VERSION/node/module.ts"; + +// import.meta.url will be the location of "this" module (like `__filename` in +// Node.js), and then serve as the root for your "package", where the +// `package.json` is expected to be, and where the `node_modules` will be used +// for resolution of packages. +const require = createRequire(import.meta.url); + +// Loads the built-in module Deno "replacement": +const path = require("path"); + +// Loads a CommonJS module (without specifying the extension): +const cjsModule = require("./my_mod"); + +// Uses Node.js resolution in `node_modules` to load the package/module. The +// package would need to be installed locally via a package management tool +// like npm: +const leftPad = require("left-pad"); +``` + +When modules are loaded via the created `require()`, they are executed in a +context which is similar to a Node.js context, which means that a lot of code +written targeting Node.js will work. diff --git a/docs/toc.json b/docs/toc.json index e6af26745a..51245c3d02 100644 --- a/docs/toc.json +++ b/docs/toc.json @@ -37,6 +37,15 @@ "import_maps": "Import maps" } }, + "npm_nodejs": { + "name": "Using npm/Node.js code", + "children": { + "std_node": "The std/node library", + "cdns": "Packages from CDNs", + "import_maps": "Using import maps", + "faqs": "Frequently asked questions" + } + }, "typescript": { "name": "Using TypeScript", "children": { diff --git a/docs/typescript/faqs.md b/docs/typescript/faqs.md index 00bc99552e..5db02cb7e4 100644 --- a/docs/typescript/faqs.md +++ b/docs/typescript/faqs.md @@ -18,10 +18,10 @@ If you are using `tsc` as stand-alone, the setting to use is `"isolatedModules"` and setting it to `true` to help ensure that your code can be properly handled by Deno. -One of the ways to deal with the extension and the lack of _magical_ resolution -is to use [import maps](../linking_to_external_code/import_maps.md) which would -allow you to specify "packages" of bare specifiers which then Deno could resolve -and load. +One of the ways to deal with the extension and the lack of Node.js non-standard +resolution logic is to use +[import maps](../linking_to_external_code/import_maps.md) which would allow you +to specify "packages" of bare specifiers which then Deno could resolve and load. ### What version(s) of TypeScript does Deno support? diff --git a/docs/typescript/overview.md b/docs/typescript/overview.md index dbf0c92862..038ea35b14 100644 --- a/docs/typescript/overview.md +++ b/docs/typescript/overview.md @@ -152,8 +152,8 @@ bypass type checking all together. ### Type resolution -One of the core design principles of Deno is to avoid "magical" resolution, and -this applies to type resolution as well. If you want to utilise JavaScript that -has type definitions (e.g. a `.d.ts` file), you have to explicitly tell Deno -about this. The details of how this is accomplished are covered in the -[Types and type declarations](./types.md) section. +One of the core design principles of Deno is to avoid non-standard module +resolution, and this applies to type resolution as well. If you want to utilise +JavaScript that has type definitions (e.g. a `.d.ts` file), you have to +explicitly tell Deno about this. The details of how this is accomplished are +covered in the [Types and type declarations](./types.md) section. diff --git a/docs/typescript/types.md b/docs/typescript/types.md index ba5462dc74..02257ff3d9 100644 --- a/docs/typescript/types.md +++ b/docs/typescript/types.md @@ -1,12 +1,12 @@ ## Types and Type Declarations -One of the design principles of Deno is no _magical_ resolution. When TypeScript -is type checking a file, it only cares about the types for the file, and the -`tsc` compiler has a lot of logic to try to resolve those types. By default, it -expects _ambiguous_ module specifiers with an extension, and will attempt to -look for the file under the `.ts` specifier, then `.d.ts`, and finally `.js` -(plus a whole other set of logic when the module resolution is set to `"node"`). -Deno deals with explicit specifiers. +One of the design principles of Deno is no non-standard module resolution. When +TypeScript is type checking a file, it only cares about the types for the file, +and the `tsc` compiler has a lot of logic to try to resolve those types. By +default, it expects _ambiguous_ module specifiers with an extension, and will +attempt to look for the file under the `.ts` specifier, then `.d.ts`, and +finally `.js` (plus a whole other set of logic when the module resolution is set +to `"node"`). Deno deals with explicit specifiers. This can cause a couple problems though. For example, let's say I want to consume a TypeScript file that has already been transpiled to JavaScript along