Compilation is unbelievably slow

[lukashavrlant]

Hi, for some reason we have performance problems when using ts-node in our mocha tests. It takes about 500 ms to compile an empty ts file. I added some console.logs to measure the time to run this very line: const output = service.getEmitOutput(fileName) and this line takes 500 ms to run even though the content of the file is an empty string (the code variable on the line 314 is empty string). Actually, all our tests files take too long to process, while the production files take a few milliseconds to compile. So any *.spec.ts takes about 500 ms to compile, a regular *.ts file takes about 20 ms. Do you have any idea what could be the root cause or how should I debug it more?

We use latest mocha, ts-node and typescript, but we tried some old versions too and the problem persists. The tsconfig.json:

{
  "compileOnSave": false,
  "compilerOptions": {
    "outDir": "dist",
    "rootDir": ".",
    "declaration": true,
    "target": "es2017",
    "lib": ["es2017"],
    "module": "commonjs",
    "moduleResolution": "node"
  },
  "exclude": [
    "node_modules",
    "dist"
  ]
}

[davidgruar]

I have also experienced this issue when running alsatian tests after upgrading to ts-node v7. I've found I can resolve the issue by specifying the --files CLI flag.

The problem seems to stem from the fact that the test framework requires each .test.ts file in turn, and each call to require appears to compile every module in that file's dependency tree from scratch. Specifying --files reverts to the old behaviour of loading all files on startup (and presumably caching them in memory).

@blakeembrey does this sound like a plausible diagnosis, and can you see any better solution?

[cspotcode]

Have you tried --transpile-only mode?

[mazswojejzony]

I've run into similar issue today. After ts-node upgrade from 7.0.1 to 8.0.1 my mocha test suite became very slow (32s vs 4s). Adding TS_NODE_TRANSPILE_ONLY=true resolved my issue. Thanks @cspotcode

[davidgruar]

--transpile-only does speed things up considerably for me. I don't want to lose the type checking though!

[lukashavrlant]

I tried all the proposed solutions. The files: true makes execution significantly faster! Turning off type checking also made things faster. I just had to turn off typeCheck and turn on transpileOnly (typeCheck: false, transpileOnly: true). This is good enough workaround, but it seems that the default behavior is still weird.

[blakeembrey]

That's intereseting, files: false is definitely meant to make things faster rather than slower. Does anyone have a large enough project that's running slow and want to either 1. investigate if the reason @davidgruar mentions appears true or 2. allow me temporary access to look into further myself?

Ideally files: false loads a single file and spiders out so it is possible that this is a slight issue with performance. It's also possible that the previous version was hitting the cache heavily, if you upgraded from v7 to v8 (v8 removed the cache because of type issues). We could also investigate adding caching back but it needs to be much smarter than the previous version to solve transitive type changes.

[garethflowers]

I've hit this issue too after updating from 7.0.1 to 8.0.1.

From what I can tell when files: false the tsconfig.json file gets changed in memory to remove the files and includes attributes.

ts-node/src/index.ts

Line 484 in 544851e

config.files = []

This seems to create an invalid config (when using TypeScript 3.2.4):

error TS18002: The 'files' list in config file 'tsconfig.json' is empty.

Removing the files and includes attributes altogether however seems to resolve the issue.

[blakeembrey]

If anyone can give me temporary access to a slow repo, let me know! I can quickly take a look into the issue and happy to sign any NDA, etc you need. If not, can I get a sense of peoples tsconfig.json that's slow - are you relying on files, include, exclude, rootDir or other to compile with TypeScript?

Let's try with reactions:

• ❤️ files
• 🚀 include
• 👀 exclude
• 😕 rootDir
• 🎉 Other

[garethflowers]

8.0.2 hasn't made any difference for me. The fact it's adding files: [] in to the tsconfig.json still kills performance.

I can't share our project unfortunately - but here's our tsconfig.json:

{
	"compileOnSave": true,
	"compilerOptions": {
		"emitDecoratorMetadata": true,
		"experimentalDecorators": true,
		"forceConsistentCasingInFileNames": true,
		"lib": [
			"es2017"
		],
		"module": "commonjs",
		"moduleResolution": "node",
		"newLine": "LF",
		"noEmitOnError": true,
		"noImplicitReturns": true,
		"noUnusedLocals": true,
		"noUnusedParameters": false,
		"outDir": "dist",
		"removeComments": true,
		"rootDir": ".",
		"sourceMap": true,
		"strict": true,
		"strictFunctionTypes": false,
		"suppressImplicitAnyIndexErrors": true,
		"target": "es6"
	}
}

[blakeembrey]

@garethflowers Are you using mocha or something else that makes the explanation in #754 (comment) make sense?

Alternatively, can you try populating files = ['one.ts file here'] so it's valid, but still overridden, and let me know if that changes anything?

[davidgruar]

@blakeembrey I've made a minimal repro at https://github.com/davidgruar/tsnode-perf-repro - please check it out and have a play. Loading just three test files with fairly few dependencies takes more than a second, and the more imports you add, the longer it takes.

[blakeembrey]

It doesn't seem like a huge overhead when you time the entire program, instead of just requires:

There is a difference, however, so for testing it might make the most sense to enable --files whereas running an application it doesn't - there's only a single input and TypeScript pre-processes everything anyway. I'll look into it more though because the difference is clear and it would be nice to bring the dynamic files number down.

[blakeembrey]

Ok, perfect. It does look like TypeScript makes 4x the number of requests to the filesystem when files: false. Looking into whether there's some issue with how TypeScript is functioning here and how this could be improved.

[jeremy-coleman]

i've found several settings contributing to slow compile times, im pretty sure the issue is related to settings that will delete the default setting of ignore: node_modules. IE: allowJs. Others that may also (just from my memory of guess/check) are removeComments, emitDecoratorMetadata, and a few others. I've also noticed that if you glob without file ext like include: src//* instead of src//*.ts it can cause slowdowns , idk if its chokadir watching fontfiles that may be in there or something. using isolatedModules is the sure-fire way to speed things up though.