A recent pr was opened up by a long time user to reuse the existing IMDS to deliver runtime configuration to an instance. While it is common to store configuration in the form of a package or json config, some configuration is best delivered at runtime such as secrets. There are also env vars such as ports or other dynamic configuration that one would not want to bake into an image and force the rebuilding of that image. So this cuts that additional step out and allows image reuse.
One of the unique things about unikernels is that the application you are deploying is a virtual machine so it probably wouldn't make much sense to do this on a normal ec2 instance running linux as there are many different programs with different users and each has their own configuration, security profile, etc., but since it's running nanos we know it's only a single application and thus injecting application specific env vars at boot time makes a lot more sense.
We've had the cloud_init klib for a while now that allows you to set env vars and download files at instance start, but that required having an external location to consume from - for instance consuming from a secret store like hashicorp vault.
Not everyone has that setup. Sometimes it might be a toy project or other times people might be handling secrets and env configuration differently. I know of one company that remounts a volume with tls certs every few hours for instance.
The neat thing about this klib is that you can inject your env vars at run-time (eg: right before the instance boots but after the image has been made).
What is the IMDS
The IMDS, or instance metadata server, is a common metadata server you'll find on every single public cloud provider. It has a number of functions including providing metadata, *ahem*, providing temporary access to other resources, and even letting the provider know that the instance has booted or is alive in the case of Azure.
Breaking in through the IMDS
The IMDS can be an excellent place to put dynamic configuration but it can also be the first window that someone breaks when trying to get onto your server. So you should be aware of some things first.
A SSRF (server side request forgery) vulnerability that can leak IMDS data can easily hand out credentials. You can make one request to get the role name and then get the credentials attached to it in another request.
AWS has two versions of IMDS - version 1 and version 2. Assuming you've found a vulnerability in an application that allows you to make http requests to the internal network (not good) version one allows unauthenticated calls directly to it, which SSRF attacks can utilize to gain access to tokens. IMDSv2 is session oriented and requires control over the type of request being made (GET, PUT, etc.) and the capability to forge request headers which makes it much harder to exploit. v2 also requires an initial request to make further requests. So there really is no good reason to use v1 on AWS, however not all clouds work like this.
Here is an example in a node app that blindly queries a user-supplied url with no sanitization. This might seem harmless at first until you realize you can now query the IMDS easily as this node app is on the server and can likely talk directly to it.
const express = require('express');
const http = require('http');
const app = express();
app.get('/woops', (req, res) => {
const userUrl = req.query.url;
http.get(userUrl, (response) => {
let data = '';
response.on('data', (chunk) => {
data += chunk;
});
response.on('end', () => {
res.send(data);
});
}).on('error', (err) => {
res.status(500).send('Error fetching data: ' + err.message);
});
});
app.listen(3000, () => {
console.log('Vulnerable server listening on port 3000');
});
http://127.0.0.1:3000/woops?url=http://169.254.169.254/latest/meta-data/The reality with SSRF is that if your app is vulnerable to that you are already kind of in deep shit.
Isn't Storing Secrets In the IMDS an Insecure Practice?
It should be noted that the IMDS typically is already handing out access for temporary security credentials so it's not so black/white on whether one should or should not use it for certain operations like this. If you are deploying to the cloud you are probably already storing secrets in an IMDS even if you don't realize it. Typically this would be locked down to the IAM role that the instance is using but can call things like s3 or find out data about the instance itself such as security groups or vpc details.
Generally the IMDS is not seen as a proper place to store secrets as as it typically doesn't have auth or uses encryption but again, it kind of already does store secrets in many cases.
First off - you should compare to other secret stores whose defaults might be just as loosey-goosey as this option is. If you're using base64 on your existing secret store - that's not encryption.
Second, just because you store a secret in a secret store, or an IMDS, doesn't mean it's safe after you retrieve it. The vast majority of secrets are decrypted and then used in plain text. We routinely see people leak out env vars that have things like database passwords in them.
Third if the IMDS gives me access to a restricted resource that's the same thing as a secret to begin with.
Differences on Various Instance MetaData Stores
There are quite a few differences amongst the various IMDS out there and this klib actually shows a handful of them.
Some IMDS base64 encode data and some do not. Some need tokens and others do not. Paths and header values can obviously differ.
For instance on AWS you'll see the token in use:
[PROVIDER_AWS] = {
.name = ss_static_init("AWS"),
.detect_path = ss_static_init("/latest/api/token"),
.detect_header_name = ss_static_init("X-aws-ec2-metadata-token-ttl-seconds"),
.detect_header_value = ss_static_init("21600"),
.detect_method = HTTP_REQUEST_METHOD_PUT,
.userdata_path = ss_static_init("/latest/user-data"),
.userdata_header_name = ss_static_init("X-aws-ec2-metadata-token"),
.userdata_header_value = sstring_null_init, /* filled with token */
.needs_base64_decode = false,
.needs_token = true,
},
but on Azure it might be:
[PROVIDER_AZURE] = {
.name = ss_static_init("Azure"),
.detect_path = ss_static_init("/metadata/instance?api-version=2021-01-01"),
.detect_header_name = ss_static_init("Metadata"),
.detect_header_value = ss_static_init("true"),
.detect_method = HTTP_REQUEST_METHOD_GET,
.userdata_path = ss_static_init("/metadata/instance/compute/userData?api-version=2021-01-01&format=text"),
.userdata_header_name = ss_static_init("Metadata"),
.userdata_header_value = ss_static_init("true"),
.needs_base64_decode = true,
.needs_token = false,
},
Using the IMDS userdata klib
From a user perspective you can provide the various env vars via simple configuration. Again - this configuration can be applied after the image is made and at instance create time (eg: 'ops instance create -c config.json')
For Example:
{
"CloudConfig": {
"UserData": "BOB=something\nTOM=something-else"
},
"Klibs": ["userdata_env", "tls"]
}
Whether or not you should store a secret in the IMDS is up to you but for things like ports or other dynamic configuration this new klib opens up quite a few new possibilites and we're excited to see what you do with it.
