joepie91
/
cvm
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
feature/node-rewrite
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '71d48f06cd'
${ noResults }
cvm/notes/notes.txt

653 lines
16 KiB
Plaintext
Raw Blame History

Research questions:
- GraphQL schema composability
- GraphQL DB integration
- Existing GraphQL -> DB adapters tend to require a network (HTTP API) layer, instead of providing a direct GraphQL integration. May be more useful to hand-roll the adapter.
- Ergonomic reusable filters for GraphQL nodes, potentially using custom directives
- How to define non-global custom types in GraphQL
- How to combine literal text-based schema definitions with programmatic schema construction, and whether this is necessary
- How to represent progress for long-running tasks in a GraphQL mutation, eg. for an "add ISO from HTTP URL" operation
Todo list:
- UI: Convert existing Pug templates to JSX/GraphQL
- Finalize the conversion of the physical drives page (totals etc.)
- GraphQL API: Add database support
- GraphQL API: Memory resources (usage, available, etc.)
- Wrappers: Add error handling to smartctl wrapper
- Wrappers: Complete LVM wrapper
- Internals: Storage pool implementations
- GraphQL API: Storage pool support
- Internals: ISO/template pool implementation
- GraphQL API: ISO/template pool support
#######################################
qemu-img create -f qcow2 test.qcow 5G
qemu-kvm -cdrom ~/Downloads/debian-8.5.0-amd64-netinst.iso -hda test.qcow -boot d -netdev user,id=user.0 -device e1000,netdev=user.0 -m 196 -localtime
# Getting Started
https://fedoraproject.org/wiki/How_to_use_qemu
https://en.wikibooks.org/wiki/QEMU/Images
https://nixos.org/wiki/QEMU_guest_with_networking_and_virtfs
https://mbharatkumar.wordpress.com/2010/10/09/qemu-getting-started/
http://www.nico.schottelius.org/blog/control-and-shutdown-qemu-kvm-vm-via-unix-socket/
http://man.cx/qemu-system-x86_64(1)
# Flags
https://wiki.gentoo.org/wiki/QEMU/Options
# Monitor
https://en.wikibooks.org/wiki/QEMU/Monitor
# QMP
https://raw.githubusercontent.com/qemu/qemu/master/docs/qmp-intro.txt
https://raw.githubusercontent.com/qemu/qemu/master/docs/qmp-spec.txt
https://raw.githubusercontent.com/qemu/qemu/master/qmp-commands.hx
https://raw.githubusercontent.com/qemu/qemu/master/docs/qmp-events.txt
http://wiki.qemu.org/QMP
https://kashyapc.com/2013/03/31/multiple-ways-to-access-qemu-machine-protocol-qmp/
https://www.npmjs.com/package/qemu-qmp
# QMP (old)
https://qemu.weilnetz.de/w64/2012/2012-06-28/qmp-commands.txt
https://lxr.missinglinkelectronics.com/qemu/qmp-commands.hx
# Networking
http://www.linux-kvm.org/page/Networking
http://wiki.qemu.org/Documentation/Networking
https://pve.proxmox.com/wiki/Network_Model
http://hyperlogos.org/page/HOWTO-kvm-vde-networking-Ubuntu-Debian-et-cetera
# Disk Images
https://alexeytorkhov.blogspot.nl/2009/09/mounting-raw-and-qcow2-vm-disk-images.html
https://edoceo.com/cli/qemu
http://libguestfs.org/
http://www.linux-kvm.org/page/Change_cdrom
# Storage Pools
https://libvirt.org/storage.html
# Virtio
http://www.linux-kvm.org/page/Boot_from_virtio_block_device
# Resource throttling
https://vpsboard.com/topic/4601-kvm-anti-abuse-how-do-you-counter-abuse-with-kvm-users/ - "cgroups cpuacct not a workable solution?"
https://vpsboard.com/topic/4601-kvm-anti-abuse-how-do-you-counter-abuse-with-kvm-users/?do=findComment&comment=66078 - "aka as each vm is a prossess use nice to limit cpu."
http://wiki.qemu.org/Features/AutoconvergeLiveMigration - dynamic CPU throttling lead...
https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg06737.html
# References
http://www.linux-kvm.org/page/HOWTO
http://www.linux-kvm.org/page/Management_Tools
https://github.com/ChoHag/nbsvm/blob/master/nbsvm
https://github.com/digitalocean/go-qemu/
# Full documentation
http://wiki.qemu.org/download/qemu-doc.html
{
"return": [{
"io-status": "ok",
"device": "ide1-cd0",
"locked": false,
"removable": true,
"inserted": {
"iops_rd": 0,
"detect_zeroes": "off",
"image": {
"virtual-size": 258998272,
"filename": "/home/sven/Downloads/debian-8.5.0-amd64-netinst.iso",
"format": "raw",
"actual-size": 259006464,
"dirty-flag": false
},
"iops_wr": 0,
"ro": true,
"node-name": "#block112",
"backing_file_depth": 0,
"drv": "raw",
"iops": 0,
"bps_wr": 0,
"write_threshold": 0,
"encrypted": false,
"bps": 0,
"bps_rd": 0,
"cache": {
"no-flush": false,
"direct": false,
"writeback": true
},
"file": "/home/sven/Downloads/debian-8.5.0-amd64-netinst.iso",
"encryption_key_missing": false
},
"tray_open": false,
"type": "unknown"
}, {
"io-status": "ok",
"device": "ide0-hd0",
"locked": false,
"removable": false,
"inserted": {
"iops_rd": 0,
"detect_zeroes": "off",
"image": {
"virtual-size": 5368709120,
"filename": "test.qcow",
"cluster-size": 65536,
"format": "qcow2",
"actual-size": 1939349504,
"format-specific": {
"type": "qcow2",
"data": {
"compat": "1.1",
"lazy-refcounts": false,
"refcount-bits": 16,
"corrupt": false
}
},
"dirty-flag": false
},
"iops_wr": 0,
"ro": false,
"node-name": "#block321",
"backing_file_depth": 0,
"drv": "qcow2",
"iops": 0,
"bps_wr": 0,
"write_threshold": 0,
"encrypted": false,
"bps": 0,
"bps_rd": 0,
"cache": {
"no-flush": false,
"direct": false,
"writeback": true
},
"file": "test.qcow",
"encryption_key_missing": false
},
"type": "unknown"
}, {
"device": "floppy0",
"locked": false,
"removable": true,
"tray_open": true,
"type": "unknown"
}, {
"device": "sd0",
"locked": false,
"removable": true,
"tray_open": false,
"type": "unknown"
}]
}
{
"return": [{
"io-status": "ok",
"device": "ide0-hd0",
"locked": false,
"removable": false,
"inserted": {
"iops_rd": 0,
"detect_zeroes": "off",
"image": {
"virtual-size": 5368709120,
"filename": "test.qcow",
"cluster-size": 65536,
"format": "qcow2",
"actual-size": 1939349504,
"format-specific": {
"type": "qcow2",
"data": {
"compat": "1.1",
"lazy-refcounts": false,
"refcount-bits": 16,
"corrupt": false
}
},
"dirty-flag": false
},
"iops_wr": 0,
"ro": false,
"node-name": "#block121",
"backing_file_depth": 0,
"drv": "qcow2",
"iops": 0,
"bps_wr": 0,
"write_threshold": 0,
"encrypted": false,
"bps": 0,
"bps_rd": 0,
"cache": {
"no-flush": false,
"direct": false,
"writeback": true
},
"file": "test.qcow",
"encryption_key_missing": false
},
"type": "unknown"
}, {
"io-status": "ok",
"device": "ide1-cd0",
"locked": false,
"removable": true,
"tray_open": false,
"type": "unknown"
}, {
"device": "floppy0",
"locked": false,
"removable": true,
"tray_open": true,
"type": "unknown"
}, {
"device": "sd0",
"locked": false,
"removable": true,
"tray_open": false,
"type": "unknown"
}]
}
{
"return": [{
"name": "query-rocker-of-dpa-groups"
}, {
"name": "query-rocker-of-dpa-flows"
}, {
"name": "query-rocker-ports"
}, {
"name": "query-rocker"
}, {
"name": "block-set-write-threshold"
}, {
"name": "x-input-send-event"
}, {
"name": "trace-event-set-state"
}, {
"name": "trace-event-get-state"
}, {
"name": "rtc-reset-reinjection"
}, {
"name": "query-acpi-ospm-status"
}, {
"name": "query-memory-devices"
}, {
"name": "query-memdev"
}, {
"name": "blockdev-change-medium"
}, {
"name": "query-named-block-nodes"
}, {
"name": "x-blockdev-insert-medium"
}, {
"name": "x-blockdev-remove-medium"
}, {
"name": "blockdev-close-tray"
}, {
"name": "blockdev-open-tray"
}, {
"name": "x-blockdev-del"
}, {
"name": "blockdev-add"
}, {
"name": "query-rx-filter"
}, {
"name": "chardev-remove"
}, {
"name": "chardev-add"
}, {
"name": "query-tpm-types"
}, {
"name": "query-tpm-models"
}, {
"name": "query-tpm"
}, {
"name": "query-target"
}, {
"name": "query-cpu-definitions"
}, {
"name": "query-machines"
}, {
"name": "device-list-properties"
}, {
"name": "qom-list-types"
}, {
"name": "change-vnc-password"
}, {
"name": "nbd-server-stop"
}, {
"name": "nbd-server-add"
}, {
"name": "nbd-server-start"
}, {
"name": "qom-get"
}, {
"name": "qom-set"
}, {
"name": "qom-list"
}, {
"name": "query-block-jobs"
}, {
"name": "query-balloon"
}, {
"name": "query-migrate-parameters"
}, {
"name": "migrate-set-parameters"
}, {
"name": "query-migrate-capabilities"
}, {
"name": "migrate-set-capabilities"
}, {
"name": "query-migrate"
}, {
"name": "query-command-line-options"
}, {
"name": "query-uuid"
}, {
"name": "query-name"
}, {
"name": "query-spice"
}, {
"name": "query-vnc-servers"
}, {
"name": "query-vnc"
}, {
"name": "query-mice"
}, {
"name": "query-status"
}, {
"name": "query-kvm"
}, {
"name": "query-pci"
}, {
"name": "query-iothreads"
}, {
"name": "query-cpus"
}, {
"name": "query-blockstats"
}, {
"name": "query-block"
}, {
"name": "query-chardev-backends"
}, {
"name": "query-chardev"
}, {
"name": "query-qmp-schema"
}, {
"name": "query-events"
}, {
"name": "query-commands"
}, {
"name": "query-version"
}, {
"name": "human-monitor-command"
}, {
"name": "qmp_capabilities"
}, {
"name": "add_client"
}, {
"name": "expire_password"
}, {
"name": "set_password"
}, {
"name": "block_set_io_throttle"
}, {
"name": "block_passwd"
}, {
"name": "query-fdsets"
}, {
"name": "remove-fd"
}, {
"name": "add-fd"
}, {
"name": "closefd"
}, {
"name": "getfd"
}, {
"name": "set_link"
}, {
"name": "balloon"
}, {
"name": "change-backing-file"
}, {
"name": "drive-mirror"
}, {
"name": "blockdev-snapshot-delete-internal-sync"
}, {
"name": "blockdev-snapshot-internal-sync"
}, {
"name": "blockdev-snapshot"
}, {
"name": "blockdev-snapshot-sync"
}, {
"name": "block-dirty-bitmap-clear"
}, {
"name": "block-dirty-bitmap-remove"
}, {
"name": "block-dirty-bitmap-add"
}, {
"name": "transaction"
}, {
"name": "block-job-complete"
}, {
"name": "block-job-resume"
}, {
"name": "block-job-pause"
}, {
"name": "block-job-cancel"
}, {
"name": "block-job-set-speed"
}, {
"name": "blockdev-backup"
}, {
"name": "drive-backup"
}, {
"name": "block-commit"
}, {
"name": "block-stream"
}, {
"name": "block_resize"
}, {
"name": "object-del"
}, {
"name": "object-add"
}, {
"name": "netdev_del"
}, {
"name": "netdev_add"
}, {
"name": "query-dump-guest-memory-capability"
}, {
"name": "dump-guest-memory"
}, {
"name": "client_migrate_info"
}, {
"name": "migrate_set_downtime"
}, {
"name": "migrate_set_speed"
}, {
"name": "query-migrate-cache-size"
}, {
"name": "migrate-start-postcopy"
}, {
"name": "migrate-set-cache-size"
}, {
"name": "migrate-incoming"
}, {
"name": "migrate_cancel"
}, {
"name": "migrate"
}, {
"name": "xen-set-global-dirty-log"
}, {
"name": "xen-save-devices-state"
}, {
"name": "ringbuf-read"
}, {
"name": "ringbuf-write"
}, {
"name": "inject-nmi"
}, {
"name": "pmemsave"
}, {
"name": "memsave"
}, {
"name": "cpu-add"
}, {
"name": "cpu"
}, {
"name": "send-key"
}, {
"name": "device_del"
}, {
"name": "device_add"
}, {
"name": "system_powerdown"
}, {
"name": "system_reset"
}, {
"name": "system_wakeup"
}, {
"name": "cont"
}, {
"name": "stop"
}, {
"name": "screendump"
}, {
"name": "change"
}, {
"name": "eject"
}, {
"name": "quit"
}]
}
-----------------
# Storage pools
- LVM
- Preallocated
- Sparse ("thin provisioning")
- File
- Preallocated (raw)
- Sparse (qcow2)
# Image pools
Make synchronization settable per image pool?
# Network pools?
--------------
Consider how to check disk space (total, used, allocated) and how to display overcommitting to the user
------
Reading filesystems:
/proc/self/mountinfo
https://github.com/coreutils/gnulib/blob/master/lib/mountlist.c
https://github.com/coreutils/coreutils/blob/master/src/df.c
Reading system info in general:
https://github.com/uutils/coreutils (Rust)
------
Schema stuff
Storage pools:
- node ID
- path (for file)
- volume group name (for LVM)
Storage space/volume:
- pool ID
- VM ID
-------
Tools
- resize2fs: resizing filesystems
------
Safety/reliability
- Lock a storage pool when a pvmove is in progress? How to detect that when it isn't done through the CVM interface?
------
MARKER:
- LVM tools abstraction (CLI wrapper)
- Storage pool implementation for LVM and file-based (test LVM via loopback devices)
--------- LVM stuff
To display available report fields:
<command, eg. pvs> -o help
To return data in JSON format:
<command, eg. pvs> --reportformat json
---------
error-chain
- exposed predicate function for "has error type somewhere in chain", for Bluebird .catch
----------
smartctl flags
PO--CK 0x0033 51 0 0 1 1 0 0 1 1
-O--CK 0x0032 50 0 0 1 1 0 0 1 0
----CK 0x0030 48 0 0 1 1 0 0 0 0
POSR-K 0x002f 47 0 0 1 0 1 1 1 1
-OSR-K 0x002e 46 0 0 1 0 1 1 1 0
POS--K 0x0027 39 0 0 1 0 0 1 1 1
-O---K 0x0022 34 0 0 1 0 0 0 1 0
---R-- 0x0008 8 0 0 0 0 1 0 0 0
K C R S O P
| | | | | |_ P prefailure warning
| | | | |__ O updated online
| | | |___ S speed/performance
| | |____ R error rate
| |_____ C event count
|______ K auto-keep
------
FIXME: Do a self-test after installation, to verify that all the parsing code works on the user's system (wrt possibly different versions of tools), and write an UnexpectedOutput/ExpectedOutputMissing handler that lets the user report any failures.
------
Security stuff:
- CSRF?
- Helmet?
- Zeroing volumes after/before use
------
# Stuff to add
Hardware -> Storage Devices
- Disk space used
- Disk space allocated (can be over 100% with thin provisioning!)
- Allocation:usage ratio
- IOPS
- Read/Write traffic
- Read/Write latency
------
lsblk
- Use `bytes` flag to get sizes in JSON output in bytes, rather than as a unit string!
Reference in New Issue View Git Blame Copy Permalink