Initial commit

master
Sven Slootweg hace 11 años
commit 46adf150e5

@ -0,0 +1,71 @@
## Disclaimer
All this code is very much hacked together to try and get things working in a reasonably reliable manner. It may have flaws, may be unpolished, or not even fully automated. Since pretty much noone uses HyperVM anymore anyway, I don't think this is too much of an issue.
**Do not under any circumstances attempt to use this code *without* first backing up all user and VPS data.** Use at your own risk.
This is going to take some attention and time to set up properly, and you'll be on your own for parts of it... but honestly, there's not really an alternative for mass HyperVM transfers.
## Caveats
* HyperVM is terrible. It may arbitrarily fail, and you may never find out why or how. Prepare to watch your migration status at regular intervals, taking over any transfers manually where necessary.
* **You cannot skip transfers.** While messing around with the database may give a similar result, it's not recommended. When a transfer fails, complete it manually and mark it as done manually (using `mark_done.py`).
* **Do not under any circumstances manually transfer a VPS to a new node during the migration process.** This will mess up the state in the migration database. Wait with manual migrations until the automated migrations have completed.
* To prevent issues, the migration towards a node will be suspended entirely when a failure happens. It will only resume after you have indicated that the transfer has been completed manually. Other nodes will continue migrating in the meantime.
* This usage guide was written afterwards. Doing this migration was largely a matter of trial and error, and I might be missing steps here. If you're stuck, you can always send an e-mail to admin@cryto.net, but I might simply not remember how I did something.
* **You should use the `admin` HyperVM user, and *not* an auxiliary account.**
* Reseller VPSes will fail to transfer and require manual transfer. The cause is apparently that a reseller VPS transfer has to be initiated from a different page, but I have not had the time to try and fix this.
## Requirements
* HyperVM.
* A HTTPd + PHP + MySQL setup.
* The Python `oursql` module.
## Data you need
1. A CSV database dump in the format `username,vpsid,currentnode,plan`.
2. A CSV database dump in the format `vpsid,email`.
3. A CSV list of VPS sizes in the format `size,vpsid`. You can gather this through running `du -sh /vz/private/*` on each node, creating one giant list out of results from all nodes, and replacing tabs with commas.
## Setup instructions for environment
1. Set up a new VPS (that will not be transfered!) with the environment as listed in the Requirements section.
2. Create a new database, and import `structure.sql` into it.
3. Clone the repository, and be sure to move everything in `public_html` to your document root for the HTTPd.
4. Edit the `.php` files in the document root to set the database configuration.
## Setup instructions for migration schedule
1. Create a file named `input.csv` with the data from the first database dump.
2. Modify sort.py. The `locations` variable holds a structure with all the nodes for the current locations. `new_servers` holds a structure defining how many new nodes exist for each 'category' in each location. In the existing script, these are marked as 12, 34, and 56, but you can use any numeric value here. If you increase or decrease the amount of categories, be sure to change the code elsewhere to reflect this. Change the code from line 43 to 48, to sort the plans into the right categories. Change the hostname in line 73 to reflect your hostname format.
3. Run sort.py, redirecting stdout to a file named `sorted.csv`. You now have your existing VPSes planned uniformly across the number of nodes you have for each category in each location.
4. Edit `schedule.py`, `emails.py`, `reorder_size.py`, `failed.py`, `status.py`, `mark_done.py`, and `run.py`, and set the correct database configuration in each.
5. Run `schedule.py`.
6. Create a new file containing the second database dump as `emails.csv`.
7. Run `emails.py`.
8. Create a new file containing the third database dump as `sizes.csv`.
9. Run `reorder_size.py`.
## Other setup stuff
1. Edit `run.py` to use the correct SMTP login details, sender address, and panel login details.
2. Edit the e-mail templates in `run.py` to reflect what you want to send to your users.
3. Edit `report.php` to set a 'reporting key' that is used by the HyperVM panel to authenticate itself to your reporting script.
4. Back up the original `livemigrate.php` and `switchserver.php` in `hypervm/bin/common/` (you'll need to restore these after the migration!).
5. Copy the patched files from the `patch` directory into the `hypervm/bin/common` directory.
6. Edit the reporting key in the patched `livemigrate.php` and `switchserver.php` files. Note that it has to be replaced in two places in each of the files!
7. Edit the hostname that the patched migration scripts should report to.
## Running the migration
* The migration process is largely automatic. You do need to pay regular attention to the status to resolve any issues.
* `run.py` will start the migration. Using the `--resume` flag will make it skip the initial notification e-mail phase. Do this when for whatever reason the script has terminated, and you wish to restart it.
* If a transfer fails (or gets stuck in the 'busy' phase), ensure that it has finished transfering and manually transfer it if required. Afterwards, run `mark_done.py` with the name of the target node as first parameter. It will assume that the currently transfering/failed VPS has finished migrating, send a success e-mail, and move on to the next.
* `status.py` will give you a human-readable overview of the current status for each node.
* `failed.py` will give you an overview of the relevant information for all transfers that are in the 'failed' state, and need manual attention.
* All sent e-mails are logged to `email.txt`.
## License
All this, except for obviously the BlueVM logo, is licensed under the [WTFPL](http://wtfpl.net/). Reuse as you wish. [Donations are very much appreciated](http://cryto.net/~joepie91/donate.html).

@ -0,0 +1,121 @@
<?php
include_once "htmllib/lib/include.php";
switchserver_main();
function switchserver_main()
{
global $argc, $argv;
global $gbl, $sgbl, $login, $ghtml;
//sleep(60);
initProgram("admin");
if ($argc === 1) {
print("Usage: $argv[0] --class= --name= --v-syncserver= \n");
exit;
}
try {
$opt = parse_opt($argv);
$param = get_variable($opt);
dprintr($param);
$class = $opt['class'];
$name = $opt['name'];
if (lx_core_lock("$class-$name.livemigrate")) {
exit;
}
$object = new $class(null, 'localhost', $name);
$object->get();
if ($object->dbaction === 'add') {
throw new lxException ("no_object", '', '');
exit;
}
if (!$object->syncserver) {
print("No_synserver...\n");
throw new lxException ("no_syncserver", '', '');
exit;
}
if ($param['syncserver'] === $object->syncserver) {
print("No Change...\n");
throw new lxException ("no_change", '', '');
exit;
}
$driverapp_old = $gbl->getSyncClass('localhost', $object->syncserver, $object->getClass());
$driverapp_new = $gbl->getSyncClass('localhost', $param['syncserver'], $object->getClass());
$oldserver = $object->syncserver;
$newserver = $param['syncserver'];
if ($driverapp_new !== $driverapp_old) {
throw new lxException ("the_drivers_are_different_in_two_servers", '', '');
}
$actualserver = getFQDNforServer($newserver);
setup_ssh_channel($oldserver, $newserver, $actualserver);
$ssh_port = db_get_value("sshconfig", $newserver, "ssh_port");
if (!$ssh_port) { $ssh_port = "22" ; }
$res = rl_exec_get(null, $oldserver, "exec_vzmigrate", array($object->vpsid, $actualserver, $ssh_port));
list($ret, $error) = $res;
if ($ret !== 0) {
throw new lxException ("vzmigrate_failed_due_to:$error", '', '');
}
$object->olddeleteflag = 'done';
$object->syncserver = $newserver;
$object->username = null;
$object->makeSureTheUserExists();
$object->setUpdateSubaction();
$object->write();
} catch (exception $e) {
print($e->getMessage());
/// hcak ahck... Chnage only the olddelete variable which is the mutex used for locking in the process of switch. The problem is we want to totally bail out if the switchserver fails. The corect way would be save after reverting the syncserve to the old value, but that's a bit risky. So we just use a hack to change only the olddeleteflag; Not a real hack.. This is the better way.
$message = "{$e->getMessage()}";
$message = str_replace("'", "", $message);
write_to_object($object, $message, $param['syncserver']);
$fullmesage = "Switch of {$object->getClass()}:{$object->nname} to {$param['syncserver']} failed due to {$e->getMessage()}";
file_get_contents("http://transfer.bluevm.com/report.php?action=failed&key=keygoeshere&server={$param['syncserver']}");
log_switch($fullmesage);
mail($login->contactemail, "Switch Failed:", "$fullmesage\n");
print("\n");
exit;
}
mail($login->contactemail, "Switch Succeeded", "Switch Succeeded {$object->getClass()}:$object->nname to {$param['syncserver']}\n");
file_get_contents("http://transfer.bluevm.com/report.php?action=done&key=keygoeshere&server={$param['syncserver']}");
}
function write_to_object($object, $message, $syncserver)
{
$sq = new Sqlite(null, $object->__table);
$message = str_replace("'", "", $message);
$sq->rawQuery("update {$object->__table} set olddeleteflag = 'Switch to $syncserver failed due to $message' where nname = '{$object->nname}'");
}

@ -0,0 +1,93 @@
<?php
include_once "htmllib/lib/include.php";
switchserver_main();
function switchserver_main()
{
global $argc, $argv;
global $gbl, $sgbl, $login, $ghtml;
//sleep(60);
initProgram("admin");
if ($argc === 1) {
print("Usage: $argv[0] --class= --name= --v-syncserver= \n");
exit;
}
try {
$opt = parse_opt($argv);
$param = get_variable($opt);
dprintr($param);
$class = $opt['class'];
$name = $opt['name'];
if (lx_core_lock("$class-$name.switchserver")) {
exit;
}
$object = new $class(null, 'localhost', $name);
$object->get();
if ($object->dbaction === 'add') {
throw new lxException ("no_object", '', '');
exit;
}
if (!$object->syncserver) {
print("No_synserver...\n");
throw new lxException ("no_syncserver", '', '');
exit;
}
if ($param['syncserver'] === $object->syncserver) {
print("No Change...\n");
throw new lxException ("no_change", '', '');
exit;
}
$driverapp_old = $gbl->getSyncClass('localhost', $object->syncserver, $object->get__table());
$driverapp_new = $gbl->getSyncClass('localhost', $param['syncserver'], $object->get__table());
if ($driverapp_new !== $driverapp_old) {
//throw new lxException ("the_drivers_are_different_in_two_servers", '', '');
}
$object->doupdateSwitchserver($param);
} catch (exception $e) {
print($e->getMessage());
/// hcak ahck... Chnage only the olddelete variable which is the mutex used for locking in the process of switch. The problem is we want to totally bail out if the switchserver fails. The corect way would be save after reverting the syncserve to the old value, but that's a bit risky. So we just use a hack to change only the olddeleteflag; Not a real hack.. This is the better way.
$message = "{$e->getMessage()}";
write_to_object($object, $message, $param['syncserver']);
$fullmesage = "Switch of {$object->get__table()}:{$object->nname} to $object->syncserver failed due to {$e->getMessage()}";
file_get_contents("http://transfer.bluevm.com/report.php?action=failed&key=keygoeshere&server={$param['syncserver']}");
log_switch($fullmesage);
mail($login->contactemail, "Switch Failed:", "$fullmesage\n");
print("\n");
exit;
}
mail($login->contactemail, "Switch Succeeded", "Switch Succeeded {$object->get__table()}:$object->nname to {$param['syncserver']}\n");
file_get_contents("http://transfer.bluevm.com/report.php?action=done&key=keygoeshere&server={$param['syncserver']}");
}
function write_to_object($object, $message, $syncserver)
{
$sq = new Sqlite(null, $object->get__table());
$sq->rawQuery("update {$object->get__table()} set olddeleteflag = 'Switch to $syncserver failed due to $message' where nname = '{$object->nname}'");
}

@ -0,0 +1,120 @@
<?php
$database = new PDO("mysql:host=localhost;dbname=transfer", "root", "");
if(empty($_GET['email']) || empty($_GET['key']))
{
die("No valid email and key specified. Please check the email you received about the migration of your servers.");
}
$statement = $database->prepare("SELECT * FROM emails WHERE `EmailAddress` = ? AND `Key` = ?");
$statement->bindValue(1, $_GET['email'], PDO::PARAM_STR);
$statement->bindValue(2, $_GET['key'], PDO::PARAM_STR);
$statement->execute();
$results = $statement->fetchAll();
if(count($results) == 0)
{
die("No valid email and key specified. Please check the email you received about the migration of your servers.");
}
$statement = $database->prepare("SELECT * FROM entries WHERE `EmailAddress` = ? ORDER BY `Finished` DESC");
$statement->bindValue(1, $_GET['email'], PDO::PARAM_STR);
$statement->execute();
$vpses = $statement->fetchAll();
?>
<!doctype html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel="stylesheet" href="style.css">
<link href='http://fonts.googleapis.com/css?family=Orienta' rel='stylesheet' type='text/css'>
</head>
<body>
<div class="header">
<img src="logo.png">
</div>
<div class="main">
<h1>VPS migration status</h1>
<table>
<tr>
<th></th>
<th>Username</th>
<th>Status</th>
</tr>
<?php
foreach($vpses as $row)
{
?>
<tr>
<?php if($row['Finished'] == "0")
{
?>
<td class="status"><span class="done">A</span> ⇒ B</td>
<?php
}
elseif($row['Finished'] == "1")
{
?>
<td class="status">A <span class="done"><img src="processing.gif"></span> B</td>
<?php
}
elseif($row['Finished'] == "2")
{
?>
<td class="status">A ⇒ <span class="done">B</span></td>
<?php
}
elseif($row['Finished'] == "3")
{
?>
<td class="status">A ⇒ B</td>
<?php
}
?>
<td><?php echo(htmlspecialchars($row["Username"])); ?></td>
<?php if($row['Finished'] == "0")
{
$my_pos = $row['Position'];
$statement = $database->prepare("SELECT * FROM servers WHERE `Host` = ?");
$statement->bindValue(1, $row['TargetNode'], PDO::PARAM_STR);
$statement->execute();
$node = $statement->fetch();
$current_pos = $node['Current'];
$real_pos = $my_pos - $current_pos;
?>
<td class="queue">Queued in position <?php echo($real_pos); ?>.</td>
<?php
}
elseif($row['Finished'] == "1")
{
?>
<td class="queue">Transferring...</td>
<?php
}
elseif($row['Finished'] == "2")
{
?>
<td class="queue">Done!</td>
<?php
}
elseif($row['Finished'] == "3")
{
?>
<td class="queue">Failed</td>
<?php
}
?>
</tr>
<?php
}
?>
</table>
</div>
</body>
</html>

Archivo binario no mostrado.

Después

Anchura:  |  Altura:  |  Tamaño: 7.0 KiB

Archivo binario no mostrado.

Después

Anchura:  |  Altura:  |  Tamaño: 847 B

@ -0,0 +1,28 @@
<?php
$db = "schedule.db";
$database = new PDO("mysql:host=localhost;dbname=transfer", "root", "");
$database->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
if($_GET['key'] == "keygoeshere")
{
if($_GET['action'] == "done")
{
$statement = $database->prepare("UPDATE entries SET `Finished` = 2 WHERE `Finished` = 1 AND `TargetNode` = ?");
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
$statement->execute();
$statement = $database->prepare("UPDATE servers SET `Busy` = 0 WHERE `Host` = ? AND `Busy` != 4");
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
$statement->execute();
}
elseif($_GET['action'] == "failed")
{
$statement = $database->prepare("UPDATE entries SET `Finished` = 3 WHERE `Finished` = 1 AND `TargetNode` = ?");
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
$statement->execute();
$statement = $database->prepare("UPDATE servers SET `Busy` = 2 WHERE `Host` = ? AND `Busy` != 4");
$statement->bindValue(1, $_GET['server'], PDO::PARAM_STR);
$statement->execute();
}
}

@ -0,0 +1,89 @@
body
{
font-family: Orienta, sans-serif;
margin: 0px;
padding: 0px;
}
.header
{
background-color: #93B5FB;
border-bottom: 1px solid #5D97FF;
padding: 12px;
}
.main
{
padding: 12px;
}
h1
{
margin-top: 4px;
}
table
{
border-spacing: 0px;
border-radius: 11px;
}
th, td
{
padding: 9px 12px;
border-top: 1px solid #929292;
border-right: 1px solid #929292;
text-align: left;
}
th:first-child
{
border-radius: 6px 0 0 0;
}
th:last-child
{
border-radius: 0 6px 0 0;
}
tr:last-child td:first-child
{
border-radius: 0 0 0 6px;
}
tr:last-child td:last-child
{
border-radius: 0 0 6px 0;
}
tr:last-child td
{
border-bottom: 1px solid #929292;
}
td:first-child, th:first-child
{
border-left: 1px solid #929292;
}
th:only-child
{
border-radius: 6px 6px 0 0;
}
td.status
{
color: silver;
font-size: 18px;
font-weight: bold;
}
td.status .done
{
color: #128310;
}
td.queue
{
}

@ -0,0 +1,26 @@
import oursql, csv, random, string
# input.csv must be in the format vpsid,email
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
reader = csv.reader(open("emails.csv", "r"))
emails = []
c = db.cursor()
for entry in reader:
vps_id, email = entry
c.execute("UPDATE entries SET `EmailAddress` = ? WHERE `VpsId` = ?", (email, vps_id))
print "Added e-mail address for VPS %s" % (vps_id)
if email not in emails:
emails.append(email)
db.commit()
for email in emails:
if email.strip() != "":
random_key = ''.join(random.choice(string.ascii_uppercase + string.digits) for x in xrange(0, 16))
c.execute("INSERT INTO emails (`EmailAddress`, `Key`) VALUES (?, ?)", (email, random_key))
db.commit()

@ -0,0 +1,23 @@
#!/usr/bin/python
import oursql
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
c = db.cursor()
c.execute("SELECT * FROM servers")
servers = c.fetchall()
for server in servers:
if server[2] == 2 or server[2] == 4:
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ?", (server[1], server[3]))
results = c.fetchall()
try:
vps = results[0]
except IndexError, e:
continue
if vps[6] == 3:
print "%s\t%s\t%s\t%s" % (server[1].ljust(26), str(vps[1]).ljust(6), vps[2].ljust(18), vps[3])

@ -0,0 +1,12 @@
#!/usr/bin/python
import oursql, sys
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
c = db.cursor()
server = sys.argv[1]
c.execute("UPDATE servers SET `Busy` = 0 WHERE `Host` = '%s'" % server)
c.execute("UPDATE entries SET `Finished` = 2 WHERE (`Finished` = 3 OR `Finished` = 1) AND `TargetNode` = '%s'" % server)

@ -0,0 +1,37 @@
import csv, re, operator, oursql
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
reader = csv.reader(open("sizes.csv", "r"))
sizes = {}
for entry in reader:
size, vpsid = entry
if re.match("[0-9]+", vpsid):
sizes[vpsid] = int(size)
#print "%s => %.2fG" % (vpsid, int(size) / 1024.0 / 1024.0)
sorted_list = sorted(sizes.iteritems(), key=operator.itemgetter(1))
c = db.cursor()
for vpsid, size in sorted_list:
c.execute("UPDATE entries SET `DiskUsage` = ? WHERE `VpsID` = ?", (size, vpsid))
c.execute("SELECT * FROM entries ORDER BY `TargetNode` ASC, `DiskUsage` ASC")
results = c.fetchall()
servers = {}
for row in results:
nodename = row[4]
try:
servers[nodename] += 1
except KeyError, e:
servers[nodename] = 1
c.execute("UPDATE entries SET `Position` = ? WHERE `Id` = ?", (servers[nodename], row[0]))

@ -0,0 +1,369 @@
#!/usr/bin/env python
import oursql, time, sys, requests, re
import smtplib
import datetime
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
mail_start = """
Hello,
As announced a short while ago, we are going to migrate your VPSes to new servers.
We've had some unforeseen technical difficulties that led to a delay, but these
have been resolved. This e-mail is to inform you that the migration process has
been started.
The following of your VPSes are affected:
%s
You can follow the queue position and progress of your VPS migrations here: %s
You will receive further notifications for every individual VPS that is being migrated.
Please ensure that you add admin@bluevm.com to your address book, so that
our e-mails do not end up in your spam folder.
Regards,
BlueVM Staff
https://bluevm.com/
"""
mail_vps_start = """
Hello,
This e-mail is to inform you that the migration of your VPS with the username %s has started.
You will receive an e-mail when this migration is finished.
You can follow the progress of all your VPS migrations at %s
Regards,
BlueVM Staff
https://bluevm.com/
"""
mail_vps_success = """
Hello,
This e-mail is to inform you that the migration of your VPS with the username %s has
finished successfully.
Your new IP addresses are:
%s
You can follow the progress of all your VPS migrations at %s
Regards,
BlueVM Staff
https://bluevm.com/
"""
mail_vps_error = """
Hello,
This e-mail is to inform you that the migration of your VPS with the username %s has
been aborted because an error has occurred. No data loss has occurred, and staff
will be looking into the issue shortly.
You can follow the progress of all your VPS migrations at %s
Regards,
BlueVM Staff
https://bluevm.com/
"""
class Panel(object):
def __init__(self, target):
self.target = target
self.sess = requests.Session()
def login(self, username, password):
page = self.sess.get("%s/login/" % self.target).text
match = re.search('<input type=hidden name=id value="([^"]+)">', page)
if match is None:
raise Exception("Not a valid HyperVM panel.")
fid = match.group(1)
error_string = "Login Unsuccessful"
post_target = "%s/htmllib/phplib/" % self.target
post_vars = {
"frm_clientname": username,
"frm_password": password,
"id": fid,
"login": "Login"
}
page = self.sess.post(post_target, data=post_vars).text
if error_string in page:
raise Exception("Login failed.")
def run_command(self, node, command):
post_target = "%s/display.php" % self.target
post_vars = {
"frm_o_o[0][class]": "pserver",
"frm_o_o[0][nname]": node,
"frm_pserver_c_ccenter_command": command,
"frm_pserver_c_ccenter_error": "",
"frm_action": "updateform",
"frm_subaction": "commandcenter",
"frm_change": "Execute"
}
page = self.sess.post(post_target, data=post_vars).text
result = re.search("<textarea[^>]+frmtextarea[^>]+>(.*?)<\/textarea>", page, re.DOTALL)
if result is not None:
return result.group(1)
def Vps(self, username):
return Vps(self, username)
class Vps(object):
def __init__(self, panel, username):
self.panel = panel
self.username = username
def get_ip_addresses(self):
get_target = "%s/display.php" % self.panel.target
get_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_o_cname": "vmipaddress_a",
"frm_action": "list",
"frm_list_refresh": "yes"
}
page = self.panel.sess.get(get_target, params=get_vars).text
ip_regex = '<td\s+class=collist\s+wrap\s+align=left\s+>\s+([0-9.]+)\s+<\/td>'
ip_list = re.findall(ip_regex, page)
self.ip_list = ip_list
return ip_list
def delete_ip_addresses(self, addresses):
post_target = "%s/display.php" % self.panel.target
post_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_accountselect": ",".join(addresses),
"frm_action": "delete",
"frm_o_cname": "vmipaddress_a",
"frm_confirmed": "yes",
"Confrm": "Confirm"
}
page = self.panel.sess.post(post_target, data=post_vars).text
def add_ip_addresses_num(self, num):
post_target = "%s/display.php" % self.panel.target
post_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_dttype[var]": "type",
"frm_dttype[val]": "npool",
"frm_o_cname": "vmipaddress_a",
"frm_vmipaddress_a_c_type": "npool",
"frm_vmipaddress_a_c_ip_num": num,
"frm_action": "add",
"frm_change": "Add"
}
page = self.panel.sess.post(post_target, data=post_vars).text
def transfer_to_live(self, target):
post_target = "%s/display.php" % self.panel.target
post_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_vps_c_syncserver": target,
"frm_action": "update",
"frm_subaction": "livemigrate",
"frm_change": "Update"
}
page = self.panel.sess.post(post_target, data=post_vars).text
if "Switching the Servers has been run in the background." not in page:
raise Exception("Transfer of %s to %s failed. Maybe an incorrect destination node was specified?" % (self.username, target))
def transfer_to(self, target):
post_target = "%s/display.php" % self.panel.target
post_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_vps_c_syncserver": target,
"frm_action": "update",
"frm_subaction": "switchserver",
"frm_change": "Update"
}
page = self.panel.sess.post(post_target, data=post_vars).text
if "Switching the Servers has been run in the background." not in page:
raise Exception("Transfer of %s to %s failed. Maybe an incorrect destination node was specified?" % (self.username, target))
def boot(self):
get_target = "%s/display.php" % self.panel.target
get_vars = {
"frm_o_o[0][class]": "vps",
"frm_o_o[0][nname]": self.username,
"frm_action": "update",
"frm_subaction": "boot"
}
page = self.panel.sess.get(get_target, params=get_vars).text
class Mailserver(object):
def __init__(self, ip, port, ssl, user, password):
if ssl == True:
self.smtp = smtplib.SMTP_SSL()
else:
self.smtp = smtplib.SMTP()
self.smtp.set_debuglevel(0)
self.smtp.connect(ip, port)
self.smtp.login(user, password)
def send(self, to, subject, body):
from_addr = "BlueVM <admin@bluevm.com>"
date = datetime.datetime.now().strftime("%d/%m/%Y %H:%M")
msg = "From: %s\nTo: %s\nSubject: %s\n\n%s\n" % (from_addr, to, subject, body)
self.smtp.sendmail(from_addr, to, msg)
def quit(self):
self.smtp.quit()
def send_mail(to, subject, body):
f = open("email.txt", "a")
f.write("To: %s\nSubject: %s\n\n%s\n\n\n" % (to, subject, body))
f.close()
mailserv = Mailserver("smtp.sendgrid.net", 465, True, "user", "pass")
mailserv.send(to, subject, body)
mailserv.quit()
def generate_url(email):
global db
c = db.cursor()
c.execute("SELECT * FROM emails WHERE `EmailAddress` = ?", (email,))
result = c.fetchall()[0]
return "http://transfer.bluevm.com/?email=%s&key=%s" % (result[1], result[2])
panel = Panel("http://manage.bluevm.com:8888")
panel.login("admin", "password")
print "Logged in."
c = db.cursor()
c.execute("SELECT * FROM emails")
emails = c.fetchall()
total = len(emails)
done = 0
try:
resumed = (sys.argv[1] == "--resume")
except IndexError, e:
resumed = False
if resumed == False:
for email in emails:
c.execute("SELECT * FROM entries WHERE `EmailAddress` = ?", (email[1],))
entries = c.fetchall()
if len(entries) > 0:
send_mail(email[1], "Migration process started", mail_start % ("\n\n".join(entry[2] for entry in entries), generate_url(email[1])))
done += 1
sys.stdout.write("Sent %d of %d announcement emails.\n" % (done, total))
while True:
c.execute("SELECT * FROM servers WHERE `Busy` = 2")
servers = c.fetchall()
for server in servers:
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ?", (server[1], server[3]))
vpses = c.fetchall()
c.execute("UPDATE servers SET `Busy` = 4 WHERE `Id` = ?", (server[0],))
if len(vpses) > 0:
send_mail(vpses[0][3], "VPS migration failed", mail_vps_error % (vpses[0][2], generate_url(vpses[0][3])))
sys.stderr.write("WARNING: Transfer of %s to %s failed! Further transfers for this server have been aborted.\n" % (vpses[0][2], server[1]))
else:
sys.stderr.write("WARNING: Transfer of VPS to %s failed! Further transfers for this server have been aborted.\n" % (server[1]))
c.execute("SELECT * FROM servers WHERE `Busy` = 0")
servers = c.fetchall()
if len(servers) > 0:
for server in servers:
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Position` = ? LIMIT 1", (server[1], server[3]))
results = c.fetchall()
if len(results) > 0:
last = results[0]
vps = panel.Vps(last[2])
ip_list = vps.get_ip_addresses()
ip_count = len(ip_list)
if ip_count > 0:
vps.delete_ip_addresses(ip_list)
sys.stderr.write("[%s] Deleted %d IP addresses: %s\n" % (last[2], ip_count, ", ".join(ip_list)))
vps.add_ip_addresses_num(ip_count)
sys.stderr.write("[%s] Added %d IP addresses from pool.\n" % (last[2], ip_count))
result = panel.run_command(server[1], "mkdir /vz/root/%s" % last[1])
try:
if result.strip() == "":
sys.stdout.write("[%s] Fixed root directory.\n" % last[2])
except AttributeError, e:
sys.stderr.write("[%s] WARNING: Could not fix root directory" % last[2])
vps.boot()
sys.stdout.write("[%s] Booted.\n" % last[2])
send_mail(last[3], "VPS migration finished", mail_vps_success % (last[2], "\n\n".join(vps.get_ip_addresses()), generate_url(last[3])))
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Finished` = 0 ORDER BY `Position` ASC LIMIT 1", (server[1],))
entries = c.fetchall()
if len(entries) == 0:
c.execute("UPDATE servers SET `Busy` = 3 WHERE `Id` = ?", (server[0],))
sys.stdout.write("Server %s finished migrating.\n" % server[1])
else:
eid = entries[0][0]
c.execute("UPDATE entries SET `Finished` = 1 WHERE `Id` = ?", (eid,))
c.execute("UPDATE servers SET `Busy` = 1, `Current` = `Current` + 1 WHERE `Id` = ?", (server[0],))
send_mail(entries[0][3], "VPS migration started", mail_vps_start % (entries[0][2], generate_url(entries[0][3])))
sys.stdout.write("[%s] Started transfer to %s\n" % (entries[0][2], entries[0][4]))
vps = panel.Vps(entries[0][2])
try:
vps.transfer_to(entries[0][4])
except Exception, e:
c.execute("UPDATE servers SET `Busy` = 4 WHERE `Host` = ?", (entries[0][4],))
c.execute("UPDATE entries SET `Finished` = 3 WHERE `Id` = ?", (entries[0][0],))
send_mail(entries[0][3], "VPS migration failed", mail_vps_error % (entries[0][2], generate_url(entries[0][3])))
sys.stderr.write("[%s] WARNING: Transfer to %s failed! Further transfers for this server have been aborted.\n" % (entries[0][2], entries[0][4]))
else:
c.execute("SELECT * FROM servers WHERE `Busy` != 3 AND `Busy` != 4")
if len(c.fetchall()) == 0:
sys.stdout.write("All migrations done. Check the log for any errors.\n")
exit(0)
time.sleep(5)

@ -0,0 +1,32 @@
import oursql, csv
# input.csv must be in the format username,vpsid,targetnode
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
reader = csv.reader(open("sorted.csv", "r"))
nodes = []
c = db.cursor()
for entry in reader:
username, vps_id, target_node = entry
c.execute("INSERT INTO entries (`VpsId`, `Username`, `TargetNode`, `Finished`) VALUES (?, ?, ?, 0)", (vps_id, username, target_node))
print "Adding VPS %s to queue (target node %s)" % (vps_id, target_node)
if target_node not in nodes:
nodes.append(target_node)
db.commit()
for node in nodes:
c.execute("INSERT INTO servers (`Host`, `Busy`, `Current`) VALUES (?, 0, 0)", (node,))
c.execute("SELECT Id FROM entries WHERE `TargetNode` = ?", (node,))
node_entries = c.fetchall()
counter = 1
for entry in node_entries:
c.execute("UPDATE entries SET `Position` = ? WHERE `Id` = ?", (counter, entry[0]))
counter += 1
db.commit()

@ -0,0 +1,76 @@
#!/usr/bin/env python
from collections import OrderedDict
import csv
reader = csv.reader(open("input.csv", "r"))
sorted_list = {}
locations = {
"IL": ["64.79.106.219", "64.79.106.220", "64.79.106.221"],
"CA": ["nitrogen.bluevm.com", "oxygen.bluevm.com", "phosphorus.bluevm.com", "carbon.bluevm.com", "helium.bluevm.com", "lithium.bluevm.com", "neon.bluevm.com", "argon.bluevm.com"],
"ATL": ["calcium.bluevm.com", "magnesium.bluevm.com", "silicon.bluevm.com", "sodium.bluevm.com"],
"TX": ["nickel.bluevm.com"]
}
new_servers = {
"IL": OrderedDict({12: 1, 34: 1, 56: 1}),
"CA": OrderedDict({12: 2, 34: 3, 56: 1}),
"ATL": OrderedDict({12: 1, 34: 2, 56: 1}),
"TX": OrderedDict({12: 1, 34: 1, 56: 1})
}
new = {}
for location, x in new_servers.iteritems():
sorted_list[location] = OrderedDict({})
sorted_list[location][12] = []
sorted_list[location][34] = []
sorted_list[location][56] = []
for vps in reader:
username, vpsid, current, plan = vps
location = ""
for loc, hosts in locations.iteritems():
if current in hosts:
location = loc
if location == "":
print "No location specified for %s" % (repr(vps))
continue
if "blue1" in plan or "blue2" in plan or "lebletspecial" in plan or "vps1" in plan or "vps2" in plan:
sorted_list[location][12].append((username, vpsid))
elif "blue3" in plan or "blue4" in plan or "vps3" in plan or "vps4" in plan:
sorted_list[location][34].append((username, vpsid))
elif "blue5" in plan or "blue6" in plan or "presale" in plan or "vps5" in plan or "vps6" in plan:
sorted_list[location][56].append((username, vpsid))
for location, items in new_servers.iteritems():
new[location] = OrderedDict({})
for category, count in items.iteritems():
for i in xrange(0, count):
new[location][i] = []
for location, items in sorted_list.iteritems():
current_server = 0
for category, vpses in items.iteritems():
total_vpses = len(vpses)
total_servers = new_servers[location][category]
per_server = total_vpses / total_servers
for i in xrange(0, total_servers):
start = per_server * i
end = (per_server * (i + 1)) - 1
if end + 2 >= total_vpses:
end = total_vpses - 1
current_server += 1
hostname = "s%d.c%d.%s.bluevm.com" % (current_server, category, location.lower())
for s in xrange(start, end + 1):
print "%s,%s,%s" % (vpses[s][0], vpses[s][1], hostname)

@ -0,0 +1,41 @@
#!/usr/bin/python
import oursql
db = oursql.connect(host="localhost", user="root", passwd="", db="transfer")
c = db.cursor()
c.execute("SELECT * FROM servers")
servers = c.fetchall()
for server in servers:
c.execute("SELECT * FROM entries WHERE `TargetNode` = ?", (server[1],))
vpses = c.fetchall()
total_vpses = len(vpses)
if server[2] == 0 or server[2] == 3:
current_vps = int(server[3])
else:
current_vps = int(server[3]) - 1
if server[2] == 0:
status = "Reported done."
elif server[2] == 1:
status = "Busy..."
elif server[2] == 2:
status = "Reported failure."
elif server[2] == 3:
status = "Finished."
elif server[2] == 4:
status = "Aborted."
if server[2] == 1:
c.execute("SELECT * FROM entries WHERE `TargetNode` = ? AND `Finished` = 1", (server[1],))
current = c.fetchall()[0]
current_vps_data = "\t%.2fG" % ((current[7] / 1024.0 / 1024),)
status = "Busy... (%s)" % current[2]
else:
current_vps_data = ""
print "%s%s%d/%d (%d%%)%s" % (server[1].ljust(25), status.ljust(25), current_vps, total_vpses, (100.0 * current_vps / total_vpses), current_vps_data)
Cargando…
Cancelar
Guardar