The OOTB NestJS configuration support leaves a lot to be desired, mainly it lacks any form of validation or schema.

If you’re reading this you probably know that having a webapp running with malformed configuration is not a good idea. The aim for this solution is that the application will fail to deploy if your configuration is wrong, as the validation will fail on boot.

Solution

Fortunately, you can roll your own solution with joi pretty easily. Simply, it allows you to inject typed config objects into your Injectables.

The following is an example of one of the configuration classes, each of the props are assigned from process.env. Note that it extends the BaseConfig class, which is below.

// database.config.ts
import { BaseConfig } from "@/config/config";
import * as Joi from "joi";

export class DatabaseConfig extends BaseConfig {
  public host = process.env.POSTGRES_HOST;
  public port = Number.parseInt(process.env.POSTGRES_PORT, 10);
  public username = process.env.POSTGRES_USERNAME;
  public password = process.env.POSTGRES_PASSWORD;
  public database = process.env.POSTGRES_DATABASE;

  public schema = Joi.object<DatabaseConfig>({
    host: Joi.string().min(1).required(),
    port: Joi.number().min(4).required(),
    username: Joi.string().min(1).required(),
    password: Joi.string().min(1).required(),
    database: Joi.string().min(1).required(),
  });
}

The BaseConfig class just handles joi schema validation. It would be best here to throw an exception that is caught by your request/response pipeline so nice errors can be returned to the user.

// config.ts
import { Logger } from "@nestjs/common";
import { AnySchema } from "joi";

export abstract class BaseConfig {
  public validate(): this {
    const validationResult = this.schema.validate(this, { allowUnknown: true });
    if (validationResult.error) {
      throw new Error(
        `Failed validating ${this.constructor.name} - ${validationResult.error.message}`,
      );
    }

    // don't pass the schema prop
    delete this.schema;

    Logger.log(
      `Loaded config ${this.constructor.name} - ${JSON.stringify(this)}`,
    );
    return this;
  }

  public abstract schema?: AnySchema;
}

This configuration can be injected into @Injectable() classes

// database.service.ts
@Injectable()
export class DatabaseService {
  constructor(private readonly config: DatabaseConfig) {
  ...

The config objects are provided by the ConfigProvider, which must be used in every module like this.

// database.module.ts
@Module({
  imports: [],
  providers: [DatabaseService, ConfigProvider(DatabaseConfig)],
  exports: [DatabaseService]
})
export class DatabaseModule {}

The config provider is pretty simple, it uses the type passed into the provider function to construct the object (which pulls the values from env env), then it calls the validation function.

// config.provider.ts
import { Provider, Type } from "@nestjs/common";
import { BaseConfig } from "./config";

export const ConfigProvider = <T extends BaseConfig>(clazz: Type<T>) => {
  return {
    provide: clazz,
    useFactory: (): T => {
      return new clazz().validate();
    },
  } as Provider;
};

Arguably the most important part of the setup, since we are using process.env, we need to use something like dotenv to load config into process.env.

import { NestFactory } from "@nestjs/core";
import * as dotenv from "dotenv";
import { AppModule } from "./app.module";

async function bootstrap() {
  dotenv.config(); // load the .env file
  const app = await NestFactory.create(AppModule, { cors: true });
  await app.listen(process.env.PORT || 3000);
}
bootstrap();

What next?

This is a good extensible starting point for most applications. You could extend this by combining it with built in NestJS ConfigService which would give you the ability to easily manage different environments, instead of using dotenv.

Further, you could create custom environment files in json, with process env overrides, for the best of both worlds.

It seems that windows is not immune to hybrid gpu pains. I recently discovered failures in Windows 11 23H2 switching from dGPU to iGPU on my Lenovo P1 Gen 5 (intel Xe / nv 3070Ti).

Behaviour

This machine is configured in a very common way, iGPU and dGPU are connected to the internal screen through a multiplexer, HDMI and USB-C ports are wired directly to the Nvidia GPU.

This means that the dGPU must be active to use external displays, but can be powered down for a portable configuration. The fact that the gpu can power down is really useful as even idle the gpu draws ~15w.

Typical behaviour in Windows for hybrid GPU setups is that the “High performance” gpu will be powered if either:

• A high-performance application is running, i.e. game
• An external display is plugged in

My experience has been that the dGPU doesn’t actually power down when the external display is disconnected, resulting in terrible battery life after unplugging my machine from displays.

Based on information from Nvidia control panels “gpu activity icon” I have been able to deduce that the dGPU isn’t powering down because there are applications running on it. To me this shows that it’s a windows issue, surprise surprise…

Monitoring

Open the Nvidia control panel - Desktop -> Display GPU Activity Icon in notification area.

Solution

Disable and re-enable the nvidia dGPU when you undock. The battery.ps1 powershell script disables the nvidia GPU and re-enables it which effectively kicks all processes running on the dGPU off. Be warned that this may unsafely kill any applications you are using that depend on the dGPU, but i’ve had no issues with things like browsers etc, ymmv.

battery.ps1

Get-PnpDevice -Class "Display" | Where-Object Manufacturer -eq "NVIDIA" | Disable-PnpDevice -Confirm:$false
Get-PnpDevice -Class "Display" | Where-Object Manufacturer -eq "NVIDIA" | Enable-PnpDevice -Confirm:$false

I used this command prompt convenience script to call the PS script manually, but for long term use you probably want to automate this based on power events or display PNP events.

battery.cmd

powershell -ExecutionPolicy Bypass -File c:\Users\jake\Desktop\battery.ps1

Result

‘convincing’ the dGPU to turn off changes this machine from having 1.5h battery life, to 3-4h (usage dependant), which I consider a massive win. Even if I am forced to run windows.

So I ran nvidia-smi to see what was going on

 $ nvidia-smi
 Failed to initialize NVML: Driver/library version mismatch
 NVML library version: 535.54

Weird, i’m not that familiar with nvidia driver packages, I assume something’s been overwritten / a partial update has occurred. Time to reinstall akmod-nvidia and reboot.

 $ sudo dnf reinstall akmod-nvidia
 ...
 # reboot... 

Still no luck, lets make sure that the nvidia kernel modules are loaded

 $ lsmod | grep nvidia
nvidia_drm             94208  13
nvidia_modeset       1560576  18 nvidia_drm
nvidia_uvm           3493888  0
nvidia              62517248  978 nvidia_uvm,nvidia_modeset
video                  73728  2 asus_wmi,nvidia_modeset

Looks fine, check dmesg for issues?

 $ sudo dmesg | grep nvidia

And I found this

[   34.453610] NVRM: API mismatch: the client has the version 535.54.03, but
               NVRM: this kernel module has the version 530.41.03.  Please
               NVRM: make sure that this kernel module and all NVIDIA driver
               NVRM: components have the same version.

Not really sure what NVRM is, but I assume it’s part of nVidia driver or cuda libs, so I reinstalled the driver, and the cuda package to no avail…

It seems NVRM is closely related to the nvidia kernel module, lets rebuild the initramfs and reboot.

 $ sudo dracut -f

Success! After rebuilding initramfs and rebooting, I have HW acceleration back, and nvidia-smi is reporting.

$ nvidia-smi
Wed Jun 28 07:42:25 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3080 Ti     Off | 00000000:08:00.0  On |                  N/A |
|  0%   49C    P8              19W / 350W |    792MiB / 12288MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

WSLg

Microsoft’s WSLg really impressed me, it gives you the ability to run GPU accelerated graphical linux applications in Windows through a RDP over a HV socket. While WSL(2) and WSLg are super cool pieces of cobbled together tech (yeah I love jank), the drawbacks I have found in WSL far outweighed the convenience of being able to launch graphical applications from windows.

Why not WSLg

• No “good” way to pass through USB devices. Forced to use USB over IP
• Performance issues
• Networking weirdness
• Slow filesystem

Ditching WSL

I expect these issues to be resolved in the future, but in the meantime, due to these drawbacks, I ended up running linux VMs in Hyper-V, accessing them over SSH. This gives almost native performance, Hyper-v is a type 1 hypervisor, in contrast to other offerings like Virutalbox (yuck, Oracle), or VMware Workstation ($$).

Graphical applications

The main drawback of running linux in Hyper-V is the terrible graphical performance in the rare case that I need to run a gui app. Tools like the Remote extension for VS Code, and Jetbrains Gateway give you the ability to have a Windows gpu accelerated IDE, which is really sending instructions to a remote machine over SSH (my linux VM in this case).

But what if you want to run other GUI applications?

XFowarding into windows

There are a few ways to do this, the best way I have found is to use a piece of commercial software called X410

Setup is reasonably straight forward, I followed the VSock guide as connecting over a Hyper-V specific mechanism seemed “nicer” to me than exposing another endpoint on my machine.

XFowarding Issues

Gnome

Gnome often had issues starting, I figured there was some incompatibility with the X410 xserver implementation. Since i’m not really using the Gnome desktop environment, just windows management, XFCE4 is a much better option.

Launching applications… sucks

Unlike WSLg, there is no nice Windows explorer integration to launch apps, to get around this you could run an application launcher like Rofi or similar. Since I will always have a terminal window open, I just wrote a script that detaches commands you type, and sends output to /dev/null

#!/bin/bash

run () {
    echo "Enter command:"
    read command
    exec $command </dev/null &>/dev/null &
    run
}

run

Hardware acceleration

Graphics performance is fine for web browsing and other non-demanding tasks. Unfortunately, there is no direct access to the GPU, this means no GPU accelerated ML tasks, or native video encode/decode.

What’s next?

I am investigating the possibility of replicating Microsofts WSLg implementation on Hyper-V VMs, stay tuned.

Node (pi0)

<<K9s-Shell>> Pod: default/defiant-defiant-kube-5768879777-6gxv8 | Container: defiant-kube 
/app # nslookup defiant-defiant-kube.default.svc.cluster.local
;; connection timed out; no servers could be reached

/app # nslookup google.com
;; connection timed out; no servers could be reached

/app # 

Host (pi0)

Last login: Sun Jul 25 05:21:24 2021 from 192.168.1.239
ubuntu@pi0:~$ nslookup google.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   google.com
Address: 142.250.66.174

ubuntu@pi0:~$ 

Node (pi1)

/app # nslookup defiant-defiant-kube.default.svc.cluster.local
Server:         10.43.0.10
Address:        10.43.0.10:53

Name:   defiant-defiant-kube.default.svc.cluster.local
Address: 10.42.3.131
Name:   defiant-defiant-kube.default.svc.cluster.local
Address: 10.42.0.114
[TRUNCATED]

/app # nslookup google.com
Server:         10.43.0.10
Address:        10.43.0.10:53

Non-authoritative answer:
Name:   google.com
Address: 142.250.204.14

/app # 

“no servers could be reached”
For some reason, Nodes can’t reach the node-local DNS server.

Known issue?
Multiple users of K3s have reported this, and it seems that it is a kernel or distro issue, see
https://github.com/k3s-io/k3s/issues/3624 and https://github.com/k3s-io/k3s/issues/1527

Temporary fix
Configuring flannel (the container network interface used by k9s) to use host-gw
The drawback to host-gw as the backend is that it requires layer2 connection between all nodes. as documented here
This isn’t a problem for my deployment as they are all connected to the same switch.

Verify you have no entry in $ cat /sys/class/thermal/cooling_device0/type
If you get back rpi-poe-fan
you can skip to editing the firmware config file

Make sure all packages are up to date
$ apt update && apt upgrade
reboot the pi

Install raspi-config package and raspi lib
$ apt install libraspberrypi-bin raspi-config

Run the raspi-config binary
$ raspi-config
select Interface Options
enable the I2c module
reboot the pi

Add the following to the firmware config file /boot/firmware/config.txt
It should be pre-popuated by many other config lines (by raspi-config)

# fan
dtoverlay=rpi-poe
dtparam=poe_fan_temp0=80000,poe_fan_temp0_hyst=2000
dtparam=poe_fan_temp1=78000,poe_fan_temp1_hyst=5000
dtparam=poe_fan_temp2=73000,poe_fan_temp2_hyst=3000
dtparam=poe_fan_temp3=70000,poe_fan_temp3_hyst=5000

You can now reboot your system, the fan should kick in at 70 degrees celsius