Initial commit
Sven Slootweg
12 years ago
commit
46adf150e5
@ -0,0 +1,71 @@
|
|||||||
|
## Disclaimer
|
||||||
|
|
||||||
|
All this code is very much hacked together to try and get things working in a reasonably reliable manner. It may have flaws, may be unpolished, or not even fully automated. Since pretty much noone uses HyperVM anymore anyway, I don't think this is too much of an issue.
|
||||||
|
|
||||||
|
**Do not under any circumstances attempt to use this code *without* first backing up all user and VPS data.** Use at your own risk.
|
||||||
|
|
||||||
|
This is going to take some attention and time to set up properly, and you'll be on your own for parts of it... but honestly, there's not really an alternative for mass HyperVM transfers.
|
||||||
|
|
||||||
|
## Caveats
|
||||||
|
|
||||||
|
* HyperVM is terrible. It may arbitrarily fail, and you may never find out why or how. Prepare to watch your migration status at regular intervals, taking over any transfers manually where necessary.
|
||||||
|
* **You cannot skip transfers.** While messing around with the database may give a similar result, it's not recommended. When a transfer fails, complete it manually and mark it as done manually (using `mark_done.py`).
|
||||||
|
* **Do not under any circumstances manually transfer a VPS to a new node during the migration process.** This will mess up the state in the migration database. Wait with manual migrations until the automated migrations have completed.
|
||||||
|
* To prevent issues, the migration towards a node will be suspended entirely when a failure happens. It will only resume after you have indicated that the transfer has been completed manually. Other nodes will continue migrating in the meantime.
|
||||||
|
* This usage guide was written afterwards. Doing this migration was largely a matter of trial and error, and I might be missing steps here. If you're stuck, you can always send an e-mail to admin@cryto.net, but I might simply not remember how I did something.
|
||||||
|
* **You should use the `admin` HyperVM user, and *not* an auxiliary account.**
|
||||||
|
* Reseller VPSes will fail to transfer and require manual transfer. The cause is apparently that a reseller VPS transfer has to be initiated from a different page, but I have not had the time to try and fix this.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
* HyperVM.
|
||||||
|
* A HTTPd + PHP + MySQL setup.
|
||||||
|
* The Python `oursql` module.
|
||||||
|
|
||||||
|
## Data you need
|
||||||
|
|
||||||
|
1. A CSV database dump in the format `username,vpsid,currentnode,plan`.
|
||||||
|
2. A CSV database dump in the format `vpsid,email`.
|
||||||
|
3. A CSV list of VPS sizes in the format `size,vpsid`. You can gather this through running `du -sh /vz/private/*` on each node, creating one giant list out of results from all nodes, and replacing tabs with commas.
|
||||||
|
|
||||||
|
## Setup instructions for environment
|
||||||
|
|
||||||
|
1. Set up a new VPS (that will not be transfered!) with the environment as listed in the Requirements section.
|
||||||
|
2. Create a new database, and import `structure.sql` into it.
|
||||||
|
3. Clone the repository, and be sure to move everything in `public_html` to your document root for the HTTPd.
|
||||||
|
4. Edit the `.php` files in the document root to set the database configuration.
|
||||||
|
|
||||||
|
## Setup instructions for migration schedule
|
||||||
|
|
||||||
|
1. Create a file named `input.csv` with the data from the first database dump.
|
||||||
|
2. Modify sort.py. The `locations` variable holds a structure with all the nodes for the current locations. `new_servers` holds a structure defining how many new nodes exist for each 'category' in each location. In the existing script, these are marked as 12, 34, and 56, but you can use any numeric value here. If you increase or decrease the amount of categories, be sure to change the code elsewhere to reflect this. Change the code from line 43 to 48, to sort the plans into the right categories. Change the hostname in line 73 to reflect your hostname format.
|
||||||
|
3. Run sort.py, redirecting stdout to a file named `sorted.csv`. You now have your existing VPSes planned uniformly across the number of nodes you have for each category in each location.
|
||||||
|
4. Edit `schedule.py`, `emails.py`, `reorder_size.py`, `failed.py`, `status.py`, `mark_done.py`, and `run.py`, and set the correct database configuration in each.
|
||||||
|
5. Run `schedule.py`.
|
||||||
|
6. Create a new file containing the second database dump as `emails.csv`.
|
||||||
|
7. Run `emails.py`.
|
||||||
|
8. Create a new file containing the third database dump as `sizes.csv`.
|
||||||
|
9. Run `reorder_size.py`.
|
||||||
|
|
||||||
|
## Other setup stuff
|
||||||
|
|
||||||
|
1. Edit `run.py` to use the correct SMTP login details, sender address, and panel login details.
|
||||||
|
2. Edit the e-mail templates in `run.py` to reflect what you want to send to your users.
|
||||||
|
3. Edit `report.php` to set a 'reporting key' that is used by the HyperVM panel to authenticate itself to your reporting script.
|
||||||
|
4. Back up the original `livemigrate.php` and `switchserver.php` in `hypervm/bin/common/` (you'll need to restore these after the migration!).
|
||||||
|
5. Copy the patched files from the `patch` directory into the `hypervm/bin/common` directory.
|
||||||
|
6. Edit the reporting key in the patched `livemigrate.php` and `switchserver.php` files. Note that it has to be replaced in two places in each of the files!
|
||||||
|
7. Edit the hostname that the patched migration scripts should report to.
|
||||||
|
|
||||||
|
## Running the migration
|
||||||
|
|
||||||
|
* The migration process is largely automatic. You do need to pay regular attention to the status to resolve any issues.
|
||||||
|
* `run.py` will start the migration. Using the `--resume` flag will make it skip the initial notification e-mail phase. Do this when for whatever reason the script has terminated, and you wish to restart it.
|
||||||
|
* If a transfer fails (or gets stuck in the 'busy' phase), ensure that it has finished transfering and manually transfer it if required. Afterwards, run `mark_done.py` with the name of the target node as first parameter. It will assume that the currently transfering/failed VPS has finished migrating, send a success e-mail, and move on to the next.
|
||||||
|
* `status.py` will give you a human-readable overview of the current status for each node.
|
||||||
|
* `failed.py` will give you an overview of the relevant information for all transfers that are in the 'failed' state, and need manual attention.
|
||||||
|
* All sent e-mails are logged to `email.txt`.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
All this, except for obviously the BlueVM logo, is licensed under the [WTFPL](http://wtfpl.net/). Reuse as you wish. [Donations are very much appreciated](http://cryto.net/~joepie91/donate.html).
|
||||||
@ -0,0 +1,121 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
include_once "htmllib/lib/include.php";
|
||||||
|
|
||||||
|
switchserver_main();
|
||||||
|
|
||||||
|
function switchserver_main()
|
||||||
|
{
|
||||||
|
|
||||||
|
global $argc, $argv;
|
||||||
|
global $gbl, $sgbl, $login, $ghtml;
|
||||||
|
|
||||||
|
//sleep(60);
|
||||||
|
initProgram("admin");
|
||||||
|
|
||||||
|
if ($argc === 1) {
|
||||||
|
print("Usage: $argv[0] --class= --name= --v-syncserver= \n");
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
try {
|
||||||
|
$opt = parse_opt($argv);
|
||||||
|
|
||||||
|
$param = get_variable($opt);
|
||||||
|
|
||||||
|
dprintr($param);
|
||||||
|
|
||||||
|
$class = $opt['class'];
|
||||||
|
$name = $opt['name'];
|
||||||
|
|
||||||
|
if (lx_core_lock("$class-$name.livemigrate")) {
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
$object = new $class(null, 'localhost', $name);
|
||||||
|
$object->get();
|
||||||
|
if ($object->dbaction === 'add') {
|
||||||
|
throw new lxException ("no_object", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!$object->syncserver) {
|
||||||
|
print("No_synserver...\n");
|
||||||
|
throw new lxException ("no_syncserver", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($param['syncserver'] === $object->syncserver) {
|
||||||
|
print("No Change...\n");
|
||||||
|
throw new lxException ("no_change", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
$driverapp_old = $gbl->getSyncClass('localhost', $object->syncserver, $object->getClass());
|
||||||
|
$driverapp_new = $gbl->getSyncClass('localhost', $param['syncserver'], $object->getClass());
|
||||||
|
|
||||||
|
$oldserver = $object->syncserver;
|
||||||
|
$newserver = $param['syncserver'];
|
||||||
|
|
||||||
|
if ($driverapp_new !== $driverapp_old) {
|
||||||
|
throw new lxException ("the_drivers_are_different_in_two_servers", '', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
$actualserver = getFQDNforServer($newserver);
|
||||||
|
|
||||||
|
setup_ssh_channel($oldserver, $newserver, $actualserver);
|
||||||
|
|
||||||
|
|
||||||
|
$ssh_port = db_get_value("sshconfig", $newserver, "ssh_port");
|
||||||
|
if (!$ssh_port) { $ssh_port = "22" ; }
|
||||||
|
|
||||||
|
$res = rl_exec_get(null, $oldserver, "exec_vzmigrate", array($object->vpsid, $actualserver, $ssh_port));
|
||||||
|
|
||||||
|
list($ret, $error) = $res;
|
||||||
|
|
||||||
|
|
||||||
|
if ($ret !== 0) {
|
||||||
|
throw new lxException ("vzmigrate_failed_due_to:$error", '', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
$object->olddeleteflag = 'done';
|
||||||
|
$object->syncserver = $newserver;
|
||||||
|
|
||||||
|
$object->username = null;
|
||||||
|
$object->makeSureTheUserExists();
|
||||||
|
$object->setUpdateSubaction();
|
||||||
|
$object->write();
|
||||||
|
|
||||||
|
|
||||||
|
} catch (exception $e) {
|
||||||
|
print($e->getMessage());
|
||||||
|
/// hcak ahck... Chnage only the olddelete variable which is the mutex used for locking in the process of switch. The problem is we want to totally bail out if the switchserver fails. The corect way would be save after reverting the syncserve to the old value, but that's a bit risky. So we just use a hack to change only the olddeleteflag; Not a real hack.. This is the better way.
|
||||||
|
|
||||||
|
$message = "{$e->getMessage()}";
|
||||||
|
|
||||||
|
|
||||||
|
$message = str_replace("'", "", $message);
|
||||||
|
write_to_object($object, $message, $param['syncserver']);
|
||||||
|
$fullmesage = "Switch of {$object->getClass()}:{$object->nname} to {$param['syncserver']} failed due to {$e->getMessage()}";
|
||||||
|
file_get_contents("http://transfer.bluevm.com/report.php?action=failed&key=keygoeshere&server={$param['syncserver']}");
|
||||||
|
log_switch($fullmesage);
|
||||||
|
|
||||||
|
mail($login->contactemail, "Switch Failed:", "$fullmesage\n");
|
||||||
|
print("\n");
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
mail($login->contactemail, "Switch Succeeded", "Switch Succeeded {$object->getClass()}:$object->nname to {$param['syncserver']}\n");
|
||||||
|
file_get_contents("http://transfer.bluevm.com/report.php?action=done&key=keygoeshere&server={$param['syncserver']}");
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
function write_to_object($object, $message, $syncserver)
|
||||||
|
{
|
||||||
|
$sq = new Sqlite(null, $object->__table);
|
||||||
|
$message = str_replace("'", "", $message);
|
||||||
|
$sq->rawQuery("update {$object->__table} set olddeleteflag = 'Switch to $syncserver failed due to $message' where nname = '{$object->nname}'");
|
||||||
|
}
|
||||||
@ -0,0 +1,93 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
include_once "htmllib/lib/include.php";
|
||||||
|
|
||||||
|
switchserver_main();
|
||||||
|
|
||||||
|
function switchserver_main()
|
||||||
|
{
|
||||||
|
|
||||||
|
global $argc, $argv;
|
||||||
|
global $gbl, $sgbl, $login, $ghtml;
|
||||||
|
|
||||||
|
//sleep(60);
|
||||||
|
initProgram("admin");
|
||||||
|
|
||||||
|
if ($argc === 1) {
|
||||||
|
print("Usage: $argv[0] --class= --name= --v-syncserver= \n");
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
try {
|
||||||
|
$opt = parse_opt($argv);
|
||||||
|
|
||||||
|
$param = get_variable($opt);
|
||||||
|
|
||||||
|
dprintr($param);
|
||||||
|
|
||||||
|
$class = $opt['class'];
|
||||||
|
$name = $opt['name'];
|
||||||
|
|
||||||
|
if (lx_core_lock("$class-$name.switchserver")) {
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
$object = new $class(null, 'localhost', $name);
|
||||||
|
$object->get();
|
||||||
|
if ($object->dbaction === 'add') {
|
||||||
|
throw new lxException ("no_object", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!$object->syncserver) {
|
||||||
|
print("No_synserver...\n");
|
||||||
|
throw new lxException ("no_syncserver", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($param['syncserver'] === $object->syncserver) {
|
||||||
|
print("No Change...\n");
|
||||||
|
throw new lxException ("no_change", '', '');
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
$driverapp_old = $gbl->getSyncClass('localhost', $object->syncserver, $object->get__table());
|
||||||
|
$driverapp_new = $gbl->getSyncClass('localhost', $param['syncserver'], $object->get__table());
|
||||||
|
|
||||||
|
if ($driverapp_new !== $driverapp_old) {
|
||||||
|
//throw new lxException ("the_drivers_are_different_in_two_servers", '', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
$object->doupdateSwitchserver($param);
|
||||||
|
} catch (exception $e) {
|
||||||
|
print($e->getMessage());
|
||||||
|
/// hcak ahck... Chnage only the olddelete variable which is the mutex used for locking in the process of switch. The problem is we want to totally bail out if the switchserver fails. The corect way would be save after reverting the syncserve to the old value, but that's a bit risky. So we just use a hack to change only the olddeleteflag; Not a real hack.. This is the better way.
|
||||||
|
|
||||||
|
$message = "{$e->getMessage()}";
|
||||||
|
|
||||||
|
|
||||||
|
write_to_object($object, $message, $param['syncserver']);
|
||||||
|
$fullmesage = "Switch of {$object->get__table()}:{$object->nname} to $object->syncserver failed due to {$e->getMessage()}";
|
||||||
|
file_get_contents("http://transfer.bluevm.com/report.php?action=failed&key=keygoeshere&server={$param['syncserver']}");
|
||||||
|
log_switch($fullmesage);
|
||||||
|
|
||||||
|
mail($login->contactemail, "Switch Failed:", "$fullmesage\n");
|
||||||
|
print("\n");
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
mail($login->contactemail, "Switch Succeeded", "Switch Succeeded {$object->get__table()}:$object->nname to {$param['syncserver']}\n");
|
||||||
|
file_get_contents("http://transfer.bluevm.com/report.php?action=done&key=keygoeshere&server={$param['syncserver']}");
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
function write_to_object($object, $message, $syncserver)
|
||||||
|
{
|
||||||
|
$sq = new Sqlite(null, $object->get__table());
|
||||||
|
$sq->rawQuery("update {$object->get__table()} set olddeleteflag = 'Switch to $syncserver failed due to $message' where nname = '{$object->nname}'");
|
||||||
|
}
|
||||||
@ -0,0 +1,120 @@
|
|||||||
|
<?php
|
||||||
|
$database = new PDO("mysql:host=localhost;dbname=transfer", "root", "");
|
||||||
|
|
||||||
|
if(empty($_GET['email']) || empty($_GET['key']))
|
||||||
|
{
|
||||||
|
die("No valid email and key specified. Please check the email you received about the migration of your servers.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$statement = $database->prepare("SELECT * FROM emails WHERE `EmailAddress` = ? AND `Key` = ?");
|
||||||
|
$statement->bindValue(1, $_GET['email'], PDO::PARAM_STR);
|
||||||
|
$statement->bindValue(2, $_GET['key'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
$results = $statement->fetchAll();
|
||||||
|
|
||||||
|
if(count($results) == 0)
|
||||||
|
{
|
||||||
|
die("No valid email and key specified. Please check the email you received about the migration of your servers.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$statement = $database->prepare("SELECT * FROM entries WHERE `EmailAddress` = ? ORDER BY `Finished` DESC");
|
||||||
|
$statement->bindValue(1, $_GET['email'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
$vpses = $statement->fetchAll();
|
||||||
|
|
||||||
|
?>
|
||||||
|
|
||||||
|
<!doctype html>
|
||||||
|
<html>
|
||||||
|
<head>
|
||||||
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
|
||||||
|
<link rel="stylesheet" href="style.css">
|
||||||
|
<link href='http://fonts.googleapis.com/css?family=Orienta' rel='stylesheet' type='text/css'>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="header">
|
||||||
|
<img src="logo.png">
|
||||||
|
</div>
|
||||||
|
<div class="main">
|
||||||
|
<h1>VPS migration status</h1>
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<th></th>
|
||||||
|
<th>Username</th>
|
||||||
|
<th>Status</th>
|
||||||
|
</tr>
|
||||||
|
<?php
|
||||||
|
foreach($vpses as $row)
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<tr>
|
||||||
|
<?php if($row['Finished'] == "0")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="status"><span class="done">A</span> ⇒ B</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "1")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="status">A <span class="done"><img src="processing.gif"></span> B</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "2")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="status">A ⇒ <span class="done">B</span></td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "3")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="status">A ⇒ B</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
?>
|
||||||
|
|
||||||
|
<td><?php echo(htmlspecialchars($row["Username"])); ?></td>
|
||||||
|
|
||||||
|
<?php if($row['Finished'] == "0")
|
||||||
|
{
|
||||||
|
$my_pos = $row['Position'];
|
||||||
|
|
||||||
|
$statement = $database->prepare("SELECT * FROM servers WHERE `Host` = ?");
|
||||||
|
$statement->bindValue(1, $row['TargetNode'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
$node = $statement->fetch();
|
||||||
|
$current_pos = $node['Current'];
|
||||||
|
|
||||||
|
$real_pos = $my_pos - $current_pos;
|
||||||
|
?>
|
||||||
|
<td class="queue">Queued in position <?php echo($real_pos); ?>.</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "1")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="queue">Transferring...</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "2")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="queue">Done!</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
elseif($row['Finished'] == "3")
|
||||||
|
{
|
||||||
|
?>
|
||||||
|
<td class="queue">Failed</td>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
?>
|
||||||
|
</tr>
|
||||||
|
<?php
|
||||||
|
}
|
||||||
|
?>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 7.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 847 B |
@ -0,0 +1,28 @@
|
|||||||
|
<?php
|
||||||
|
$db = "schedule.db";
|
||||||
|
|
||||||
|
$database = new PDO("mysql:host=localhost;dbname=transfer", "root", "");
|
||||||
|
|
||||||
|
$database->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
|
||||||
|
if($_GET['key'] == "keygoeshere")
|
||||||
|
{
|
||||||
|
if($_GET['action'] == "done")
|
||||||
|
{
|
||||||
|
$statement = $database->prepare("UPDATE entries SET `Finished` = 2 WHERE `Finished` = 1 AND `TargetNode` = ?");
|
||||||
|
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
$statement = $database->prepare("UPDATE servers SET `Busy` = 0 WHERE `Host` = ? AND `Busy` != 4");
|
||||||
|
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
}
|
||||||
|
elseif($_GET['action'] == "failed")
|
||||||
|
{
|
||||||
|
$statement = $database->prepare("UPDATE entries SET `Finished` = 3 WHERE `Finished` = 1 AND `TargetNode` = ?");
|
||||||
|
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
$statement = $database->prepare("UPDATE servers SET `Busy` = 2 WHERE `Host` = ? AND `Busy` != 4");
|
||||||
|
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
|
||||||
|
$statement->execute();
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,89 @@
|
|||||||
|
body
|
||||||
|
{
|
||||||
|
font-family: Orienta, sans-serif;
|
||||||
|
margin: 0px;
|
||||||
|
padding: 0px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.header
|
||||||
|
{
|
||||||
|
background-color: #93B5FB;
|
||||||
|
border-bottom: 1px solid #5D97FF;
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.main
|
||||||
|
{
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1
|
||||||
|
{
|
||||||
|
margin-top: 4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
table
|
||||||
|
{
|
||||||
|
border-spacing: 0px;
|
||||||
|
border-radius: 11px;
|
||||||
|
}
|
||||||
|
|
||||||
|
th, td
|
||||||
|
{
|
||||||
|
padding: 9px 12px;
|
||||||
|
border-top: 1px solid #929292;
|
||||||
|
border-right: 1px solid #929292;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
th:first-child
|
||||||
|
{
|
||||||
|
border-radius: 6px 0 0 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
th:last-child
|
||||||
|
{
|
||||||
|
border-radius: 0 6px 0 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
tr:last-child td:first-child
|
||||||
|
{
|
||||||
|
border-radius: 0 0 0 6px;
|
||||||
|
}
|
||||||
|
|
||||||
|
tr:last-child td:last-child
|
||||||
|
{
|
||||||
|
border-radius: 0 0 6px 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
tr:last-child td
|
||||||
|
{
|
||||||
|
border-bottom: 1px solid #929292;
|
||||||
|
}
|
||||||
|
|
||||||
|
td:first-child, th:first-child
|
||||||
|
{
|
||||||
|
border-left: 1px solid #929292;
|
||||||
|
}
|
||||||
|
|
||||||
|
th:only-child
|
||||||
|
{
|
||||||
|
border-radius: 6px 6px 0 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
td.status
|
||||||
|
{
|
||||||
|
color: silver;
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
|
||||||
|
td.status .done
|
||||||
|
{
|
||||||
|
color: #128310;
|
||||||
|
}
|
||||||
|
|
||||||
|
td.queue
|
||||||
|
{
|
||||||
|
|
||||||
|
}
|
||||||
@ -0,0 +1,26 @@
|
|||||||
|
import oursql, csv, random, string
|
||||||
|
|
||||||
|
# input.csv must be in the format vpsid,email
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
reader = csv.reader(open("emails.csv", "r"))
|
||||||
|
|
||||||
|
emails = []
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
for entry in reader:
|
||||||
|
vps_id, email = entry
|
||||||
|
c.execute("UPDATE entries SET `EmailAddress` = ? WHERE `VpsId` = ?", (email, vps_id))
|
||||||
|
print "Added e-mail address for VPS %s" % (vps_id)
|
||||||
|
|
||||||
|
if email not in emails:
|
||||||
|
emails.append(email)
|
||||||
|
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
for email in emails:
|
||||||
|
if email.strip() != "":
|
||||||
|
random_key = ''.join(random.choice(string.ascii_uppercase + string.digits) for x in xrange(0, 16))
|
||||||
|
c.execute("INSERT INTO emails (`EmailAddress`, `Key`) VALUES (?, ?)", (email, random_key))
|
||||||
|
|
||||||
|
db.commit()
|
||||||
@ -0,0 +1,23 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
|
||||||
|
import oursql
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM servers")
|
||||||
|
servers = c.fetchall()
|
||||||
|
|
||||||
|
for server in servers:
|
||||||
|
if server[2] == 2 or server[2] == 4:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ?", (server[1], server[3]))
|
||||||
|
results = c.fetchall()
|
||||||
|
|
||||||
|
try:
|
||||||
|
vps = results[0]
|
||||||
|
except IndexError, e:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if vps[6] == 3:
|
||||||
|
print "%s\t%s\t%s\t%s" % (server[1].ljust(26), str(vps[1]).ljust(6), vps[2].ljust(18), vps[3])
|
||||||
|
|
||||||
@ -0,0 +1,12 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
|
||||||
|
import oursql, sys
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
server = sys.argv[1]
|
||||||
|
|
||||||
|
c.execute("UPDATE servers SET `Busy` = 0 WHERE `Host` = '%s'" % server)
|
||||||
|
c.execute("UPDATE entries SET `Finished` = 2 WHERE (`Finished` = 3 OR `Finished` = 1) AND `TargetNode` = '%s'" % server)
|
||||||
|
|
||||||
@ -0,0 +1,37 @@
|
|||||||
|
import csv, re, operator, oursql
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
reader = csv.reader(open("sizes.csv", "r"))
|
||||||
|
|
||||||
|
sizes = {}
|
||||||
|
|
||||||
|
for entry in reader:
|
||||||
|
size, vpsid = entry
|
||||||
|
|
||||||
|
if re.match("[0-9]+", vpsid):
|
||||||
|
sizes[vpsid] = int(size)
|
||||||
|
#print "%s => %.2fG" % (vpsid, int(size) / 1024.0 / 1024.0)
|
||||||
|
|
||||||
|
sorted_list = sorted(sizes.iteritems(), key=operator.itemgetter(1))
|
||||||
|
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
for vpsid, size in sorted_list:
|
||||||
|
c.execute("UPDATE entries SET `DiskUsage` = ? WHERE `VpsID` = ?", (size, vpsid))
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM entries ORDER BY `TargetNode` ASC, `DiskUsage` ASC")
|
||||||
|
results = c.fetchall()
|
||||||
|
|
||||||
|
servers = {}
|
||||||
|
|
||||||
|
for row in results:
|
||||||
|
nodename = row[4]
|
||||||
|
|
||||||
|
try:
|
||||||
|
servers[nodename] += 1
|
||||||
|
except KeyError, e:
|
||||||
|
servers[nodename] = 1
|
||||||
|
|
||||||
|
c.execute("UPDATE entries SET `Position` = ? WHERE `Id` = ?", (servers[nodename], row[0]))
|
||||||
|
|
||||||
|
|
||||||
@ -0,0 +1,369 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
|
||||||
|
import oursql, time, sys, requests, re
|
||||||
|
import smtplib
|
||||||
|
import datetime
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
|
||||||
|
mail_start = """
|
||||||
|
Hello,
|
||||||
|
|
||||||
|
As announced a short while ago, we are going to migrate your VPSes to new servers.
|
||||||
|
We've had some unforeseen technical difficulties that led to a delay, but these
|
||||||
|
have been resolved. This e-mail is to inform you that the migration process has
|
||||||
|
been started.
|
||||||
|
|
||||||
|
The following of your VPSes are affected:
|
||||||
|
|
||||||
|
%s
|
||||||
|
|
||||||
|
You can follow the queue position and progress of your VPS migrations here: %s
|
||||||
|
|
||||||
|
You will receive further notifications for every individual VPS that is being migrated.
|
||||||
|
Please ensure that you add admin@bluevm.com to your address book, so that
|
||||||
|
our e-mails do not end up in your spam folder.
|
||||||
|
|
||||||
|
Regards,
|
||||||
|
BlueVM Staff
|
||||||
|
https://bluevm.com/
|
||||||
|
"""
|
||||||
|
|
||||||
|
mail_vps_start = """
|
||||||
|
Hello,
|
||||||
|
|
||||||
|
This e-mail is to inform you that the migration of your VPS with the username %s has started.
|
||||||
|
You will receive an e-mail when this migration is finished.
|
||||||
|
|
||||||
|
You can follow the progress of all your VPS migrations at %s
|
||||||
|
|
||||||
|
Regards,
|
||||||
|
BlueVM Staff
|
||||||
|
https://bluevm.com/
|
||||||
|
"""
|
||||||
|
|
||||||
|
mail_vps_success = """
|
||||||
|
Hello,
|
||||||
|
|
||||||
|
This e-mail is to inform you that the migration of your VPS with the username %s has
|
||||||
|
finished successfully.
|
||||||
|
|
||||||
|
Your new IP addresses are:
|
||||||
|
|
||||||
|
%s
|
||||||
|
|
||||||
|
You can follow the progress of all your VPS migrations at %s
|
||||||
|
|
||||||
|
Regards,
|
||||||
|
BlueVM Staff
|
||||||
|
https://bluevm.com/
|
||||||
|
"""
|
||||||
|
|
||||||
|
mail_vps_error = """
|
||||||
|
Hello,
|
||||||
|
|
||||||
|
This e-mail is to inform you that the migration of your VPS with the username %s has
|
||||||
|
been aborted because an error has occurred. No data loss has occurred, and staff
|
||||||
|
will be looking into the issue shortly.
|
||||||
|
|
||||||
|
You can follow the progress of all your VPS migrations at %s
|
||||||
|
|
||||||
|
Regards,
|
||||||
|
BlueVM Staff
|
||||||
|
https://bluevm.com/
|
||||||
|
"""
|
||||||
|
|
||||||
|
class Panel(object):
|
||||||
|
def __init__(self, target):
|
||||||
|
self.target = target
|
||||||
|
self.sess = requests.Session()
|
||||||
|
|
||||||
|
def login(self, username, password):
|
||||||
|
page = self.sess.get("%s/login/" % self.target).text
|
||||||
|
match = re.search('<input type=hidden name=id value="([^"]+)">', page)
|
||||||
|
|
||||||
|
if match is None:
|
||||||
|
raise Exception("Not a valid HyperVM panel.")
|
||||||
|
|
||||||
|
fid = match.group(1)
|
||||||
|
|
||||||
|
error_string = "Login Unsuccessful"
|
||||||
|
post_target = "%s/htmllib/phplib/" % self.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_clientname": username,
|
||||||
|
"frm_password": password,
|
||||||
|
"id": fid,
|
||||||
|
"login": "Login"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.sess.post(post_target, data=post_vars).text
|
||||||
|
|
||||||
|
if error_string in page:
|
||||||
|
raise Exception("Login failed.")
|
||||||
|
|
||||||
|
def run_command(self, node, command):
|
||||||
|
post_target = "%s/display.php" % self.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_o_o[0][class]": "pserver",
|
||||||
|
"frm_o_o[0][nname]": node,
|
||||||
|
"frm_pserver_c_ccenter_command": command,
|
||||||
|
"frm_pserver_c_ccenter_error": "",
|
||||||
|
"frm_action": "updateform",
|
||||||
|
"frm_subaction": "commandcenter",
|
||||||
|
"frm_change": "Execute"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.sess.post(post_target, data=post_vars).text
|
||||||
|
result = re.search("<textarea[^>]+frmtextarea[^>]+>(.*?)<\/textarea>", page, re.DOTALL)
|
||||||
|
|
||||||
|
if result is not None:
|
||||||
|
return result.group(1)
|
||||||
|
|
||||||
|
def Vps(self, username):
|
||||||
|
return Vps(self, username)
|
||||||
|
|
||||||
|
class Vps(object):
|
||||||
|
def __init__(self, panel, username):
|
||||||
|
self.panel = panel
|
||||||
|
self.username = username
|
||||||
|
|
||||||
|
def get_ip_addresses(self):
|
||||||
|
get_target = "%s/display.php" % self.panel.target
|
||||||
|
get_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_o_cname": "vmipaddress_a",
|
||||||
|
"frm_action": "list",
|
||||||
|
"frm_list_refresh": "yes"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.get(get_target, params=get_vars).text
|
||||||
|
|
||||||
|
ip_regex = '<td\s+class=collist\s+wrap\s+align=left\s+>\s+([0-9.]+)\s+<\/td>'
|
||||||
|
ip_list = re.findall(ip_regex, page)
|
||||||
|
self.ip_list = ip_list
|
||||||
|
|
||||||
|
return ip_list
|
||||||
|
|
||||||
|
def delete_ip_addresses(self, addresses):
|
||||||
|
post_target = "%s/display.php" % self.panel.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_accountselect": ",".join(addresses),
|
||||||
|
"frm_action": "delete",
|
||||||
|
"frm_o_cname": "vmipaddress_a",
|
||||||
|
"frm_confirmed": "yes",
|
||||||
|
"Confrm": "Confirm"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.post(post_target, data=post_vars).text
|
||||||
|
|
||||||
|
def add_ip_addresses_num(self, num):
|
||||||
|
post_target = "%s/display.php" % self.panel.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_dttype[var]": "type",
|
||||||
|
"frm_dttype[val]": "npool",
|
||||||
|
"frm_o_cname": "vmipaddress_a",
|
||||||
|
"frm_vmipaddress_a_c_type": "npool",
|
||||||
|
"frm_vmipaddress_a_c_ip_num": num,
|
||||||
|
"frm_action": "add",
|
||||||
|
"frm_change": "Add"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.post(post_target, data=post_vars).text
|
||||||
|
|
||||||
|
def transfer_to_live(self, target):
|
||||||
|
post_target = "%s/display.php" % self.panel.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_vps_c_syncserver": target,
|
||||||
|
"frm_action": "update",
|
||||||
|
"frm_subaction": "livemigrate",
|
||||||
|
"frm_change": "Update"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.post(post_target, data=post_vars).text
|
||||||
|
|
||||||
|
if "Switching the Servers has been run in the background." not in page:
|
||||||
|
raise Exception("Transfer of %s to %s failed. Maybe an incorrect destination node was specified?" % (self.username, target))
|
||||||
|
|
||||||
|
def transfer_to(self, target):
|
||||||
|
post_target = "%s/display.php" % self.panel.target
|
||||||
|
post_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_vps_c_syncserver": target,
|
||||||
|
"frm_action": "update",
|
||||||
|
"frm_subaction": "switchserver",
|
||||||
|
"frm_change": "Update"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.post(post_target, data=post_vars).text
|
||||||
|
|
||||||
|
if "Switching the Servers has been run in the background." not in page:
|
||||||
|
raise Exception("Transfer of %s to %s failed. Maybe an incorrect destination node was specified?" % (self.username, target))
|
||||||
|
|
||||||
|
def boot(self):
|
||||||
|
get_target = "%s/display.php" % self.panel.target
|
||||||
|
get_vars = {
|
||||||
|
"frm_o_o[0][class]": "vps",
|
||||||
|
"frm_o_o[0][nname]": self.username,
|
||||||
|
"frm_action": "update",
|
||||||
|
"frm_subaction": "boot"
|
||||||
|
}
|
||||||
|
|
||||||
|
page = self.panel.sess.get(get_target, params=get_vars).text
|
||||||
|
|
||||||
|
class Mailserver(object):
|
||||||
|
def __init__(self, ip, port, ssl, user, password):
|
||||||
|
if ssl == True:
|
||||||
|
self.smtp = smtplib.SMTP_SSL()
|
||||||
|
else:
|
||||||
|
self.smtp = smtplib.SMTP()
|
||||||
|
|
||||||
|
self.smtp.set_debuglevel(0)
|
||||||
|
self.smtp.connect(ip, port)
|
||||||
|
self.smtp.login(user, password)
|
||||||
|
|
||||||
|
def send(self, to, subject, body):
|
||||||
|
from_addr = "BlueVM <admin@bluevm.com>"
|
||||||
|
date = datetime.datetime.now().strftime("%d/%m/%Y %H:%M")
|
||||||
|
msg = "From: %s\nTo: %s\nSubject: %s\n\n%s\n" % (from_addr, to, subject, body)
|
||||||
|
|
||||||
|
self.smtp.sendmail(from_addr, to, msg)
|
||||||
|
|
||||||
|
def quit(self):
|
||||||
|
self.smtp.quit()
|
||||||
|
|
||||||
|
def send_mail(to, subject, body):
|
||||||
|
f = open("email.txt", "a")
|
||||||
|
f.write("To: %s\nSubject: %s\n\n%s\n\n\n" % (to, subject, body))
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
mailserv = Mailserver("smtp.sendgrid.net", 465, True, "user", "pass")
|
||||||
|
mailserv.send(to, subject, body)
|
||||||
|
mailserv.quit()
|
||||||
|
|
||||||
|
def generate_url(email):
|
||||||
|
global db
|
||||||
|
c = db.cursor()
|
||||||
|
c.execute("SELECT * FROM emails WHERE `EmailAddress` = ?", (email,))
|
||||||
|
result = c.fetchall()[0]
|
||||||
|
return "http://transfer.bluevm.com/?email=%s&key=%s" % (result[1], result[2])
|
||||||
|
|
||||||
|
panel = Panel("http://manage.bluevm.com:8888")
|
||||||
|
panel.login("admin", "password")
|
||||||
|
print "Logged in."
|
||||||
|
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM emails")
|
||||||
|
emails = c.fetchall()
|
||||||
|
total = len(emails)
|
||||||
|
done = 0
|
||||||
|
|
||||||
|
try:
|
||||||
|
resumed = (sys.argv[1] == "--resume")
|
||||||
|
except IndexError, e:
|
||||||
|
resumed = False
|
||||||
|
|
||||||
|
if resumed == False:
|
||||||
|
for email in emails:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `EmailAddress` = ?", (email[1],))
|
||||||
|
entries = c.fetchall()
|
||||||
|
|
||||||
|
if len(entries) > 0:
|
||||||
|
send_mail(email[1], "Migration process started", mail_start % ("\n\n".join(entry[2] for entry in entries), generate_url(email[1])))
|
||||||
|
|
||||||
|
done += 1
|
||||||
|
|
||||||
|
sys.stdout.write("Sent %d of %d announcement emails.\n" % (done, total))
|
||||||
|
|
||||||
|
while True:
|
||||||
|
c.execute("SELECT * FROM servers WHERE `Busy` = 2")
|
||||||
|
servers = c.fetchall()
|
||||||
|
|
||||||
|
for server in servers:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ?", (server[1], server[3]))
|
||||||
|
vpses = c.fetchall()
|
||||||
|
|
||||||
|
c.execute("UPDATE servers SET `Busy` = 4 WHERE `Id` = ?", (server[0],))
|
||||||
|
|
||||||
|
if len(vpses) > 0:
|
||||||
|
send_mail(vpses[0][3], "VPS migration failed", mail_vps_error % (vpses[0][2], generate_url(vpses[0][3])))
|
||||||
|
sys.stderr.write("WARNING: Transfer of %s to %s failed! Further transfers for this server have been aborted.\n" % (vpses[0][2], server[1]))
|
||||||
|
else:
|
||||||
|
sys.stderr.write("WARNING: Transfer of VPS to %s failed! Further transfers for this server have been aborted.\n" % (server[1]))
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM servers WHERE `Busy` = 0")
|
||||||
|
servers = c.fetchall()
|
||||||
|
|
||||||
|
if len(servers) > 0:
|
||||||
|
for server in servers:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ? LIMIT 1", (server[1], server[3]))
|
||||||
|
results = c.fetchall()
|
||||||
|
|
||||||
|
if len(results) > 0:
|
||||||
|
last = results[0]
|
||||||
|
|
||||||
|
vps = panel.Vps(last[2])
|
||||||
|
ip_list = vps.get_ip_addresses()
|
||||||
|
ip_count = len(ip_list)
|
||||||
|
|
||||||
|
if ip_count > 0:
|
||||||
|
vps.delete_ip_addresses(ip_list)
|
||||||
|
sys.stderr.write("[%s] Deleted %d IP addresses: %s\n" % (last[2], ip_count, ", ".join(ip_list)))
|
||||||
|
|
||||||
|
vps.add_ip_addresses_num(ip_count)
|
||||||
|
sys.stderr.write("[%s] Added %d IP addresses from pool.\n" % (last[2], ip_count))
|
||||||
|
|
||||||
|
result = panel.run_command(server[1], "mkdir /vz/root/%s" % last[1])
|
||||||
|
|
||||||
|
try:
|
||||||
|
if result.strip() == "":
|
||||||
|
sys.stdout.write("[%s] Fixed root directory.\n" % last[2])
|
||||||
|
except AttributeError, e:
|
||||||
|
sys.stderr.write("[%s] WARNING: Could not fix root directory" % last[2])
|
||||||
|
|
||||||
|
vps.boot()
|
||||||
|
sys.stdout.write("[%s] Booted.\n" % last[2])
|
||||||
|
|
||||||
|
send_mail(last[3], "VPS migration finished", mail_vps_success % (last[2], "\n\n".join(vps.get_ip_addresses()), generate_url(last[3])))
|
||||||
|
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Finished` = 0 ORDER BY `Position` ASC LIMIT 1", (server[1],))
|
||||||
|
entries = c.fetchall()
|
||||||
|
|
||||||
|
if len(entries) == 0:
|
||||||
|
c.execute("UPDATE servers SET `Busy` = 3 WHERE `Id` = ?", (server[0],))
|
||||||
|
sys.stdout.write("Server %s finished migrating.\n" % server[1])
|
||||||
|
else:
|
||||||
|
eid = entries[0][0]
|
||||||
|
c.execute("UPDATE entries SET `Finished` = 1 WHERE `Id` = ?", (eid,))
|
||||||
|
c.execute("UPDATE servers SET `Busy` = 1, `Current` = `Current` + 1 WHERE `Id` = ?", (server[0],))
|
||||||
|
|
||||||
|
send_mail(entries[0][3], "VPS migration started", mail_vps_start % (entries[0][2], generate_url(entries[0][3])))
|
||||||
|
sys.stdout.write("[%s] Started transfer to %s\n" % (entries[0][2], entries[0][4]))
|
||||||
|
|
||||||
|
vps = panel.Vps(entries[0][2])
|
||||||
|
|
||||||
|
try:
|
||||||
|
vps.transfer_to(entries[0][4])
|
||||||
|
except Exception, e:
|
||||||
|
c.execute("UPDATE servers SET `Busy` = 4 WHERE `Host` = ?", (entries[0][4],))
|
||||||
|
c.execute("UPDATE entries SET `Finished` = 3 WHERE `Id` = ?", (entries[0][0],))
|
||||||
|
send_mail(entries[0][3], "VPS migration failed", mail_vps_error % (entries[0][2], generate_url(entries[0][3])))
|
||||||
|
sys.stderr.write("[%s] WARNING: Transfer to %s failed! Further transfers for this server have been aborted.\n" % (entries[0][2], entries[0][4]))
|
||||||
|
else:
|
||||||
|
c.execute("SELECT * FROM servers WHERE `Busy` != 3 AND `Busy` != 4")
|
||||||
|
|
||||||
|
if len(c.fetchall()) == 0:
|
||||||
|
sys.stdout.write("All migrations done. Check the log for any errors.\n")
|
||||||
|
exit(0)
|
||||||
|
|
||||||
|
|
||||||
|
time.sleep(5)
|
||||||
@ -0,0 +1,32 @@
|
|||||||
|
import oursql, csv
|
||||||
|
|
||||||
|
# input.csv must be in the format username,vpsid,targetnode
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
reader = csv.reader(open("sorted.csv", "r"))
|
||||||
|
|
||||||
|
nodes = []
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
for entry in reader:
|
||||||
|
username, vps_id, target_node = entry
|
||||||
|
c.execute("INSERT INTO entries (`VpsId`, `Username`, `TargetNode`, `Finished`) VALUES (?, ?, ?, 0)", (vps_id, username, target_node))
|
||||||
|
print "Adding VPS %s to queue (target node %s)" % (vps_id, target_node)
|
||||||
|
|
||||||
|
if target_node not in nodes:
|
||||||
|
nodes.append(target_node)
|
||||||
|
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
for node in nodes:
|
||||||
|
c.execute("INSERT INTO servers (`Host`, `Busy`, `Current`) VALUES (?, 0, 0)", (node,))
|
||||||
|
|
||||||
|
c.execute("SELECT Id FROM entries WHERE `TargetNode` = ?", (node,))
|
||||||
|
node_entries = c.fetchall()
|
||||||
|
counter = 1
|
||||||
|
|
||||||
|
for entry in node_entries:
|
||||||
|
c.execute("UPDATE entries SET `Position` = ? WHERE `Id` = ?", (counter, entry[0]))
|
||||||
|
counter += 1
|
||||||
|
|
||||||
|
db.commit()
|
||||||
@ -0,0 +1,76 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
from collections import OrderedDict
|
||||||
|
import csv
|
||||||
|
|
||||||
|
reader = csv.reader(open("input.csv", "r"))
|
||||||
|
sorted_list = {}
|
||||||
|
|
||||||
|
locations = {
|
||||||
|
"IL": ["64.79.106.219", "64.79.106.220", "64.79.106.221"],
|
||||||
|
"CA": ["nitrogen.bluevm.com", "oxygen.bluevm.com", "phosphorus.bluevm.com", "carbon.bluevm.com", "helium.bluevm.com", "lithium.bluevm.com", "neon.bluevm.com", "argon.bluevm.com"],
|
||||||
|
"ATL": ["calcium.bluevm.com", "magnesium.bluevm.com", "silicon.bluevm.com", "sodium.bluevm.com"],
|
||||||
|
"TX": ["nickel.bluevm.com"]
|
||||||
|
}
|
||||||
|
|
||||||
|
new_servers = {
|
||||||
|
"IL": OrderedDict({12: 1, 34: 1, 56: 1}),
|
||||||
|
"CA": OrderedDict({12: 2, 34: 3, 56: 1}),
|
||||||
|
"ATL": OrderedDict({12: 1, 34: 2, 56: 1}),
|
||||||
|
"TX": OrderedDict({12: 1, 34: 1, 56: 1})
|
||||||
|
}
|
||||||
|
|
||||||
|
new = {}
|
||||||
|
|
||||||
|
for location, x in new_servers.iteritems():
|
||||||
|
sorted_list[location] = OrderedDict({})
|
||||||
|
sorted_list[location][12] = []
|
||||||
|
sorted_list[location][34] = []
|
||||||
|
sorted_list[location][56] = []
|
||||||
|
|
||||||
|
for vps in reader:
|
||||||
|
username, vpsid, current, plan = vps
|
||||||
|
|
||||||
|
location = ""
|
||||||
|
|
||||||
|
for loc, hosts in locations.iteritems():
|
||||||
|
if current in hosts:
|
||||||
|
location = loc
|
||||||
|
|
||||||
|
if location == "":
|
||||||
|
print "No location specified for %s" % (repr(vps))
|
||||||
|
continue
|
||||||
|
|
||||||
|
if "blue1" in plan or "blue2" in plan or "lebletspecial" in plan or "vps1" in plan or "vps2" in plan:
|
||||||
|
sorted_list[location][12].append((username, vpsid))
|
||||||
|
elif "blue3" in plan or "blue4" in plan or "vps3" in plan or "vps4" in plan:
|
||||||
|
sorted_list[location][34].append((username, vpsid))
|
||||||
|
elif "blue5" in plan or "blue6" in plan or "presale" in plan or "vps5" in plan or "vps6" in plan:
|
||||||
|
sorted_list[location][56].append((username, vpsid))
|
||||||
|
|
||||||
|
for location, items in new_servers.iteritems():
|
||||||
|
new[location] = OrderedDict({})
|
||||||
|
|
||||||
|
for category, count in items.iteritems():
|
||||||
|
for i in xrange(0, count):
|
||||||
|
new[location][i] = []
|
||||||
|
|
||||||
|
for location, items in sorted_list.iteritems():
|
||||||
|
current_server = 0
|
||||||
|
|
||||||
|
for category, vpses in items.iteritems():
|
||||||
|
total_vpses = len(vpses)
|
||||||
|
total_servers = new_servers[location][category]
|
||||||
|
per_server = total_vpses / total_servers
|
||||||
|
|
||||||
|
for i in xrange(0, total_servers):
|
||||||
|
start = per_server * i
|
||||||
|
end = (per_server * (i + 1)) - 1
|
||||||
|
|
||||||
|
if end + 2 >= total_vpses:
|
||||||
|
end = total_vpses - 1
|
||||||
|
|
||||||
|
current_server += 1
|
||||||
|
hostname = "s%d.c%d.%s.bluevm.com" % (current_server, category, location.lower())
|
||||||
|
|
||||||
|
for s in xrange(start, end + 1):
|
||||||
|
print "%s,%s,%s" % (vpses[s][0], vpses[s][1], hostname)
|
||||||
@ -0,0 +1,41 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
|
||||||
|
import oursql
|
||||||
|
|
||||||
|
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
|
||||||
|
c = db.cursor()
|
||||||
|
|
||||||
|
c.execute("SELECT * FROM servers")
|
||||||
|
servers = c.fetchall()
|
||||||
|
|
||||||
|
for server in servers:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ?", (server[1],))
|
||||||
|
vpses = c.fetchall()
|
||||||
|
|
||||||
|
total_vpses = len(vpses)
|
||||||
|
|
||||||
|
if server[2] == 0 or server[2] == 3:
|
||||||
|
current_vps = int(server[3])
|
||||||
|
else:
|
||||||
|
current_vps = int(server[3]) - 1
|
||||||
|
|
||||||
|
if server[2] == 0:
|
||||||
|
status = "Reported done."
|
||||||
|
elif server[2] == 1:
|
||||||
|
status = "Busy..."
|
||||||
|
elif server[2] == 2:
|
||||||
|
status = "Reported failure."
|
||||||
|
elif server[2] == 3:
|
||||||
|
status = "Finished."
|
||||||
|
elif server[2] == 4:
|
||||||
|
status = "Aborted."
|
||||||
|
|
||||||
|
if server[2] == 1:
|
||||||
|
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Finished` = 1", (server[1],))
|
||||||
|
current = c.fetchall()[0]
|
||||||
|
current_vps_data = "\t%.2fG" % ((current[7] / 1024.0 / 1024),)
|
||||||
|
status = "Busy... (%s)" % current[2]
|
||||||
|
else:
|
||||||
|
current_vps_data = ""
|
||||||
|
|
||||||
|
print "%s%s%d/%d (%d%%)%s" % (server[1].ljust(25), status.ljust(25), current_vps, total_vpses, (100.0 * current_vps / total_vpses), current_vps_data)
|
||||||
Loading…
Reference in New Issue