Where Does Cloud-Init Store Datasource (ds) Data Locally?

The main cloud-init state directory is:

/var/lib/cloud/

Typical structure:

/var/lib/cloud/
├── data/
├── instance/
├── instances/
├── seed/
└── sem/

Important Directories

Current Instance Directory

/var/lib/cloud/instance/

This is usually a symlink to:

/var/lib/cloud/instances/<instance-id>/

Example:

/var/lib/cloud/instances/i-0abcd1234/

This directory contains:

user-data
vendor-data
meta-data
datasource information
rendered scripts
network configuration

Most Useful Files

Datasource Information

/var/lib/cloud/instance/datasource

You can also inspect:

/var/lib/cloud/instance/obj.pkl

Instance Metadata Cache

One of the most important files:

/var/lib/cloud/instance/instance-data.json

Sensitive version:

/var/lib/cloud/instance/instance-data-sensitive.json

These files may contain:

instance ID
region
availability zone
hostname
network details
cloud provider metadata
datasource content

View it with:

jq . /var/lib/cloud/instance/instance-data.json

Datasource Cache

Cloud-init stores datasource selection and status under:

/var/lib/cloud/data/

Common files:

/var/lib/cloud/data/
├── instance-id
├── previous-instance-id
├── previous-datasource
├── result.json
└── status.json

User-Data Storage

Raw user-data is typically stored at:

/var/lib/cloud/instance/user-data.txt

And sometimes:

/var/lib/cloud/instance/user-data.txt.i

Rendered Scripts

Cloud-init generated scripts are stored in:

/var/lib/cloud/instance/scripts/

Example:

/var/lib/cloud/instance/scripts/runcmd

Network Configuration Cache

Datasource-provided network configuration may be rendered into:

Ubuntu

/etc/netplan/

RHEL / CentOS

/etc/sysconfig/network-scripts/

Cloud-init also keeps an internal copy:

/var/lib/cloud/instance/network-config

Quick Ways to Check the Datasource

cloud-id

or:

cloud-init query ds

View All Datasource Data

cloud-init query --all

or directly:

jq . /var/lib/cloud/instance/instance-data.json

Runtime Copy of Instance Metadata

Cloud-init also exposes runtime metadata at:

/run/cloud-init/instance-data.json

Completely Reset Cloud-Init Cache

cloud-init clean --logs
rm -rf /var/lib/cloud/*

Then reboot the VM.

This forces cloud-init to behave like first boot again.

Basic Status & Debugging

# Check cloud-init status
cloud-init status

# Wait until cloud-init finishes
cloud-init status --wait

# Show detailed status
cloud-init status --long

# View cloud-init version
cloud-init --version

# Analyze boot timing
cloud-init analyze blame

# Show boot stages timing
cloud-init analyze show

# Dump full analyze data
cloud-init analyze dump

Logs

# Main cloud-init log
tail -f /var/log/cloud-init.log

# Cloud-init output log
tail -f /var/log/cloud-init-output.log

# Search for errors
grep -i error /var/log/cloud-init.log

# Search warnings
grep -i warn /var/log/cloud-init.log

Query Instance Metadata

# Show all instance metadata
cloud-init query --all

# Query datasource info
cloud-init query ds

# Query instance ID
cloud-init query v1.instance_id

# Query hostname
cloud-init query v1.local_hostname

# Query cloud platform
cloud-init query platform

# Pretty-print JSON
cloud-init query --format yaml

# Example: get region
cloud-init query v1.region

User Data

# Show raw user-data
cloud-init query userdata

# Show vendor-data
cloud-init query vendordata

# Show merged config
cloud-init query combined_cloud_config

Some datasources do not expose userdata directly.

Re-run Cloud-Init

# Clean cloud-init state
cloud-init clean

# Clean logs too
cloud-init clean --logs

# Re-run init stage
cloud-init init

# Re-run modules
cloud-init modules --mode=config

# Re-run final modules
cloud-init modules --mode=final

Single Module Execution

# Run a single module
cloud-init single --name cc_set_hostname

# Example: rerun ssh module
cloud-init single --name cc_ssh

Config & Schema Validation

# Validate cloud-config YAML
cloud-init schema --config-file myconfig.yaml

# Dump current merged config
cloud-init devel net-convert --debug

Network Troubleshooting

# Show generated network config
cat /etc/netplan/50-cloud-init.yaml

# Check datasource
cloud-id

# View datasource detection
cloud-init query ds

# Check rendered network config
cloud-init query ds.network_json

Datasource Investigation

# Detect datasource
cloud-id

# Show datasource details
cloud-init query ds

# Show datasource class
cloud-init query ds.meta_data

# List datasource attempts
grep datasource /var/log/cloud-init.log

Useful File Locations

# Main config
/etc/cloud/cloud.cfg

# Additional configs
/etc/cloud/cloud.cfg.d/

# Instance data
/var/lib/cloud/

# Logs
/var/log/cloud-init.log
/var/log/cloud-init-output.log

# Generated instance scripts
/var/lib/cloud/instance/scripts/

Common Troubleshooting Flow

# 1. Check status
cloud-init status --long

# 2. Check logs
less /var/log/cloud-init.log

# 3. Verify datasource
cloud-id
cloud-init query ds

# 4. Verify user-data
cloud-init query userdata

# 5. Clean and rerun if needed
cloud-init clean --logs
reboot

Quick One-Liners

Was cloud-init successful?

cloud-init status

Did user-data run?

grep -i userdata /var/log/cloud-init.log

Did package install fail?

grep -i "failed" /var/log/cloud-init.log

Find executed scripts

find /var/lib/cloud/instance/scripts/

Check current datasource

cloud-id

Helpful Systemd Commands

# Cloud-init services
systemctl status cloud-init
systemctl status cloud-config
systemctl status cloud-final

# Boot ordering
systemd-analyze critical-chain cloud-init.service

Typical Cloud-Init Stages

init-local
init
config
final

You can manually rerun stages:

cloud-init modules --mode=init
cloud-init modules --mode=config
cloud-init modules --mode=final

Example: Full Reset & Re-Provision

sudo cloud-init clean --logs
sudo rm -rf /var/lib/cloud/*
sudo reboot

Use carefully — this makes the VM behave like first boot again.