 ZPop
|
read my profile
sign my guestbook
Name: larry
Country: United States
State: Massachusetts
Interests: Sending jokes to Zach. Being nosey. Walking Brewster. Music.
Expertise: Being proud of my kids. It's a tough job, but someone's gotta do it.
Occupation: Research and development
Industry: Research
Message: message meEmail: email me
Website: visit my website
Member Since:
1/19/2002
|
|
|  “spiritual shortcuts” from nakedpastor Posted via web from Slothoughts Hypolite | | |
| That's the prescription we have been given for swine flu, or as we now like to call it, so as not to offend the sensibilities of our porcine brothers and sisters, H1N1. Of course what we were really told was to wash our hands endlessly, for a full 20 seconds at a time. If you actually look at a watch for 20 seconds, you'll see that is a rather long time. So long, in fact, that you can't even come close to guessing it. Thus, you need to find something to say or do while you're washing your hands all that time. Something that takes a full 20 seconds to say or do. That something turns out to be two complete Hail Marys. Now, given that I was born and bred a Protestant, one of the more liberal Protestant denominations actually (née UPC USA; currently UCC), how would I know about Hail Marys? Well, it seems that when I was in 7th or 8th grade, I used to listen to the radio a lot. I had rather a good one, a Hammarlund HQ100 A. I liked changing stations on both AM and shortwave to see what I could find. I would listen to stations from Buffalo, Nashville, Cleveland, Boston, Toronto, Moscow, Quito. You name it, I probably heard it at one time or another. One day, I chanced upon a group reciting the rosary. Over and over again they chanted: Hail Mary, full of grace,
the Lord is with thee.
Blessed art thou among women.
Blessed is the fruit of thy womb,
Jesus Christ, Our Lord. Holy Mary, mother of God,
Pray for us sinners,
Now and at the hour of our death.
Amen. I may have this a bit wrong. Seventh or eighth grade was a long time ago. Whatever, when I started washing my hands, and looking at my watch to see if I'd done it for twenty seconds, those words from long ago popped into my head. I discovered that, when said at a measured pace, two full Hail Marys, at least as I had remembered them, took up twenty seconds. So that's how we combat swine flu H1N1: we spend quality time at the sink saying our Hail Marys. The Pope would be proud of me. Posted via email from Slothoughts Hypolite | | |
| I said something in my class a couple of weeks ago about visiting a Buddhist temple in Wakefield along with a comment on the art. That was a mistake, of sorts, in that my teacher, Catherine, then wanted me to post a link or two (which means work for me). Anyway, my friend, Roy, has a flickr page where he has posted photos of some of the art work. His photo stream has a picture of the dharma hall with the hanging lotus lanterns.
Every year, they tear down and remake all these lanterns. My attempts at paper art never got past folding origami cranes, so I found the description of the lantern making process quite fascinating. Roy blogged the process in steps: one, two, three, four. Posted via email from Slothoughts Hypolite | | |
| Some of the better off people in my neighborhood, like the guy who appears in a popular public TV production (and the hottie across and slightly down the street from him), can afford to hire "landscapers" to take care of their yards. Thus on a regular basis, Brewster and I, who spend quite a bit of time walking around the neighborhood, checking out the sights and smells, see platoons of little brown people scurrying around these people's lawns.
Yes, I know that sounds a bit racist. I also know that I have a son who has worked as a landscaper, so the stereotype isn't completely true. None-the-less, the vast majority of landscapers in my neighborhood are somewhat more swarthy than your average Anglo Saxon person, whose family has lived in the U.S. for half a dozen generations...or more. As nearly as I can tell, they aren't speaking English to each other either. I have no problem with that. That's just how things are in the suburbs north of Boston.
So it was quite interesting to me when YSOP assigned my mission trip group to be a landscape crew one day. After a long subway ride out from the center of Washington, D.C., and a bit of a walk, we found ourselves in front of a building managed by Mr. Barry. He has a last name, but I forget it. He likes to be called Barry, but, given that he's clearly the boss of his domain, he is "Mr. Barry" to us lesser folks.
Mr. Barry manages a transition home, Milestone Place, housing about 35 people for an organization called Community Family Life Service. He is quite strict with them: no loud noise; no drugs; no alcohol; no overnight visitors; clean up after yourself in common areas; etc. Mr. Barry doesn't take crap from anyone. If you leave your dirty dishes in the sink, figuring you'll do them later, you'll find them in the dumpster.
But, Mr. Barry had a problem. He doesn't have much of a budget for upkeep. If you don't keep a place up, the tenants soon get slack and things go to the dogs...so to speak. Mr. Barry, however, is a very smart man. He has found a solution to his problem: volunteers.
Yup, Mr. Barry gets volunteers to come out to his place and do maintenance of various sorts. Sometimes it's painting, sometimes repair, sometimes cleaning. For my group, mostly teenage girls, we were to do "yard work". We began with mowing the lawn and trimming the edges. We also did quite a bit of weeding and some general trash pick up in the parking lot. As I was pushing the lawn mower around, I suddenly realized how smart Mr. Barry was. Whereas my neighbors, like the public TV personality (and the hottie across and slightly down the street from him), have to pay good money to get little brown people to keep up their yards, Mr. Barry can get a platoon of little white people to keep up his yard for free.
He had something over 1700 volunteers last year, and expects to have even more this coming year to keep things neat and tidy for him. Don't take my word for it. Go visit Mr. Barry. You'll see a picture of the girls and me on his wall, along with pictures of the other several thousand volunteers he's had work for him over the past few years. Just don't forget to bring a rake, broom, or paint brush with you. Posted via email from Slothoughts Hypolite | | |
| by ngungo and lgpiper
Introduction
When the Ma.gnolia site went down a few weeks ago, most people's data were irretrievably lost. One can learn two lessons from this issue. One is that a robust, off-site back up plan is necessary. The other is that back-ups necessarily need be redundant. Thus, if one back-up becomes corrupt or incapacitated in any way, the data are still available via the second back up. We deal with the first of these two problems: developing a routine that ensures regular back up. Our concern is with moderate sized data bases. We are developing a few small web applications: Monpage.com and bookMarkR.us. They may well grow slowly, but they will never grow at all if we don't protect our users' data from the beginning.
Background
We wish to develop a generalized backup routine. The language of the back up is of little consequence. We have chosen to program our back-up routine in PHP. The core of the back up will be a function that looks something like the following:
function dirBK($from, $dest) {
// This function will recursively copy all files and sub directories
// from a single directory, $from, on a hosting server
// to a single directory, $dest, on a remote storage disk.
// rsync seems the best candidate for the file-copying protocol.
}
If there are a lot data in a huge number of directories--e.g. one directory per monpage account, or one directory per site that each account monitors--, using rsync is not without its own problems. It can tie up the i/o ports on the server, then websites--anything else for that matter--on the hosting server would become temporarily unavailable. All activity would freeze whilst waiting for rsync to finish. Depending on the amount of data, this shut-down could take up to several hours. One way to solve the problem is to run the back up in piecemeal fashion, backing up one small directory at a time, with a pause in between each back-up segment. Thus, the function dirBK() must incorporate a loop which spreads the many small back-ups over an extended period of time--e.g. hours, days....
Another problem with rsync arises where, under a non-ideal scenario such as broadband disruption, rsync hangs and does nothing. We must, therefore, find a solution for this problem too.
Given the above considerations, we now have three major tasks, each of which may include some subtasks:
- Developing the dirBK($from, $dest) routine.
- Developing a general driver to run this routine in a loop.
- Scheduling the driver (e.g. running the back-up routine regularly using cron.)
This article only deals with the first two tasks because the scheduling task is server specific and also because, once the general routine functions properly, automatic scheduling is relatively simple to implement.
Back-up Routine Development
Developing dirBK is rsync specific and secondary to the topic here, so we will treat it last. Just be aware that it can be disrupted and hold up the whole grand scheme based on our driver routine. We will first describe the driver in simple terms, than add detail as we go along.
We begin with the variable, $dirs, which is an array of all directories that need to be backed up. The driver routine then becomes:
foreach($dirs as $dir) {
$from = 'path/to/dir/'.$dir;
$dest = 'path/to/destination/'.$dir;
dirBK($from, $dest);
sleep(5);
}
This seems to be a good approach, backing up each directory, one at a time, and pausing few seconds in between. The pausing time could be a tuning parameter, depending on the efficiency of the i/o. What might still cause problems, however, is the situation in which dirBK() function halts in the middle of the file transfer process. One solution might be if we could somehow run each individual directory back up as a independent process. Thus, if one transfer breaks down, others will still be able to continue normally. To do this, we need to implement process spawning, using the pcntl_fork() function. Our modified routine now becomes:
foreach($dirs as $dir) {
$from = 'path/to/dir/'.$dir;
$dest = 'path/to/destination/'.$dir;
$pid = pcntl_fork(); // spawn child process
if (!$pid) { // a child process returns zero
dirBK($from, $dest);
exit;
}
sleep(5);
}
We further need to check 2 things. First, if we will want to exclude some directories. For example, in a normal directory scanning we should exclude the "dot" and "double dot" directories. Secondly, we must be sure that what we are backing up is, indeed, a directory. So the next version of the routine is:
foreach($dirs as $dir) {
if ($dir == '.') continue; //exclude '.' and '..' directories
if ($dir == '..') continue;
$from = 'path/to/dir/'.$dir;
if (is_dir($from)) { //make sure array element is a directory
$dest = 'path/to/destination/'.$dir;
$pid = pcntl_fork(); // spawn child process
if (!$pid) { // a child process returns zero
dirBK($from, $dest);
exit;
}
}
sleep(5);
}
At the end, the driver quits and all independent backup processes, running as the driver's children, also quit, including those that hung up for whatever reason. The command, sleep(5), gives some breathing room between each backup increment. Sometime, however, five seconds will not be long enough to ensure that the last cycle is complete. Thus, we give it a a little extra time, perhaps an extra 30 seconds.
Above, we specified an array of the directories to back up, $dirs. There are a number of ways to populate this array. If all of the directories are in the rootBK directory, we can use the scandir() function to populate the array. Our basic routine is now as follows:
<?php
$dirs = scandir('path/to/rootBK/'); //populate the array, $dirs
foreach($dirs as $dir) {
if ($dir == '.') continue; //exclude '.' and '..' directories
if ($dir == '..') continue;
$from = 'path/to/dir/'.$dir;
if (is_dir($from)) { //make sure array element is a directory
$dest = 'path/to/destination/'.$dir;
$pid = pcntl_fork(); // spawn child process
if (!$pid) { // a child process returns zero
dirBK($from, $dest);
exit;
}
}
sleep(5);
}
sleep(30);
?>
It now remains to develop the dirBK routine. The whole routine can just a system call to the rsync command:
rsync -r /path/to/existingsite/ username@newsite.com:/path/to/newsite/
It may be necessary to extend this with a few more parameters to specify a password file:
rsync -azq --delete -e "ssh -i /users/home/myaccount/.ssh/ss" /path/to/backups/ myaccount@myaccount.strongspace.com:/home/myaccount/backups
We won't go into the details of the rsync command, but you can find them on line at the following urls:
We will use the second method above, i.e. with the password file specified. To break up the command into more comprehensible components, we have,
$rsync = '/path/to/rsync -azq --delete -e "ssh -i /.ssh/ss"';
$from = '/path/to/backups/';
$remote = 'myaccount@myaccount.strongspace.com:/home/myaccount/backups';
Then, as mentioned above, we embed these variables into a php system call, remembering that we need a space between the three strings, $rsync, $from, and $remote:
system($rsync. ' ' .$from. ' ' .$remote);
Placing this call into the main scheme produces the final version:
<?php
$rsync = '/path/to/rsync -azq --delete -e "ssh -i /.ssh/ss"';
$remote = 'myaccount@myaccount.strongspace.com:/home/myaccount/backups';
$dirs = scandir('path/to/rootBK/');
foreach($dirs as $dir) {
if ($dir == '.') continue;
if ($dir == '..') continue;
$from = 'path/to/dir/'.$dir;
if (is_dir($from)) {
$dest = $remote.$dir;
$pid = pcntl_fork();
if (!$pid) {
system($rsync. ' ' .$from. ' ' .$dest);
// echo $dir;
exit;
}
}
sleep(5);
}
sleep(30);
?>
The echo $dir command was added for debugging purposes. To begin, one might wish to comment out the system call and un-comment the echo line. Then when the routine appears to be processing the expected directories properly, reverse the comments so that the full back-up is working.
If the name the file containing our routine backUp.php, we can run it manually from the command line, making sure to be in the proper working directory:
# php5 backUp.php
Sometimes it is necessary to specify the full path to the php command, and also the php.ini file. In that case the command might look as follows:
/usr/local/bin/php -c /users/home/userName/etc/php5/ /users/home/userName/backUp.php
The specifics, naturally, depend on one's particular server configuration.
Once the routine works as intended from the command line, then one should write a cron job to automate the process. That is a subject for another time. Posted via email from Slothoughts Hypolite | | |
|
