Search This Blog
Wednesday, 7 March 2012
Display number of Apache connections
netstat -an |grep :80 |wc -l
Friday, 2 March 2012
Ubuntu/Backtrack eth0/wlan0 Missing
My good deed for the week was to offer my laptop as a replacement for one with a screwed VGA port.
Dual boot with Ubuntu 11.10 & BackTrack (Ubuntu). Both failed to connect to the network and had created eth1 & wlan1 interfaces.
Google took me to http://www.orzeszek.org/blog/2010/07/25/fix-missing-eth0-when-cloning-ubuntu-vmware-virtual-machines/
Its discussing the same problem, but in a virtual environment. The conclusion is the same in that the MAC address has changed and Ubuntu stores this as a reference in :
'Yeah just switch the disks' - Linux is clever, it will work it out. #FAIL
Dual boot with Ubuntu 11.10 & BackTrack (Ubuntu). Both failed to connect to the network and had created eth1 & wlan1 interfaces.
Google took me to http://www.orzeszek.org/blog/2010/07/25/fix-missing-eth0-when-cloning-ubuntu-vmware-virtual-machines/
Its discussing the same problem, but in a virtual environment. The conclusion is the same in that the MAC address has changed and Ubuntu stores this as a reference in :
/etc/udev/rules.d/70-persistent-net.rulesDelete the file, reboot, problem fixed.
Tuesday, 21 February 2012
I think I broke Google
A tweet from @MarkMaceSmith asking :
A quick look and he was right (kind of).. But not just his images, EVERYONES!
http://www.paulregan.co.uk/m/viewAlbum?uname=110372774069389594216&aid=5492124562868999953&start=0
Shows his gallery on my domain. Note the the Picasa Mobile URL (/m/) which also gave me the answer.
I have masked the Picasa URL using an Apache rewriteRule on my site :
I can probably fix this with robots.txt, but ..
Had me wondering what the hell !?@paulmregan Why have you got hundreds of my images on your website Paul Regan?
A quick look and he was right (kind of).. But not just his images, EVERYONES!
http://www.paulregan.co.uk/m/viewAlbum?uname=110372774069389594216&aid=5492124562868999953&start=0
Shows his gallery on my domain. Note the the Picasa Mobile URL (/m/) which also gave me the answer.
I have masked the Picasa URL using an Apache rewriteRule on my site :
RewriteRule ^m/(.*) http://picasaweb.google.com/m/$1 [NC,P,L]It appears the Google spiders came along and followed this all the way through Picasa's structure. I don't know how deep but Google site:www.paulregan.co.uk/m/ currently returns >250K links and every one of them works!
I can probably fix this with robots.txt, but ..
- Should I?
- Have I inadvertently discovered SEO gold?
- Will I be struck off and sent to web-master jail/gaol? (I probably should be)
#twitter
Monday, 20 February 2012
Linux Clonezilla "Error: no argument specified"
Ubuntu 10.04 LTS cloned from an identical friend using Clonezilla.
When you boot theres a quick flash of an error, in this case SSD disks so too quick to read, and your presented with the GRUB menu.
Selecting the current kernel threw out :
Fix :
When you boot theres a quick flash of an error, in this case SSD disks so too quick to read, and your presented with the GRUB menu.
Selecting the current kernel threw out :
Error: no argument specifiedAll the grub config files appeared correct.
Fix :
- apt-get remove grub-pc
- apt-get remove grub-common
- rm /boot/grub/grub.cfg
- apt-get install grub-common
- apt-get install grub-pc
- Install grub to the disk to contain MBR, in my case /dev/sda /
Labels:
Linux
Wednesday, 15 February 2012
Puppy Linux Full Install
- Download and create the Puppy linux LiveCD/LIveUSB
- Boot the PC using #1
- Follow the options to configure the screen res and then the locale options
- Prepare the disk, if you need. Menu -> System -> GParted. Delete/create the partitions as you need and restart the PC. Don't save settings
- Install the files to disk. Menu -> Setup -> Puppy universal installer. Follow the prompts to select your local disk.
- Install Grub. Menu -> System -> Grub legacy boot loader. Follow the prompts and when asked select your local disk, eg /dev/sda1 and make sure you select destination to MBR.
Puppy Linux installed on 9 year old Sony Vaio. I thank you.
Labels:
Linux
Wednesday, 8 February 2012
Crowbar and OpenStack Pt Deux
As a follow up to part 1 which left me with an 'installed' environment but no networking I thought I should post a follow up.
Since I last messed around a new version of crowbar (1.2) had been released so that was used. A nice addition is rather than research/guess the OpenStack barclamp install order they are now nicely arranged on the Crowbar admin panel.
I also came to the conclusion that my networking 'issues' were a result of both me trying to install on vSphere, which has no support for native VLAN's and by complicating things by trying to shoe horn the networking into my corp range.
So I build a little lab which consisted of anything not nailed to a desk, I could find lying around, or that no-body would notice missing.
Switch - vlans created with SVI ports as needed for default gateways and all ports 802.1q to the nodes.
Crowbar install went fine using Pt1 to fill in the missing details.
The Swift install failed until I added some more storage (those cutting edge USB disks on top) and restarted the Crowbar install from scratch.
My first attempt at building a instance failed. The instance remained in status of 'building'.
To debug
Use the Nova console to check which node the image has been deployed to and then grep the local /var/log/nova/nova-compute.log for ERROR.
This is a result of the crappy hardware I'm using so I changed the nova barclamp to 'qemu' and bingo, deployed instance - well, sometimes. It seems pretty hit and miss, which I can't yet trace. I suspect I could still be suffering from the Heath Robinson lab I've put together. Pt.3 :)
Some useful bits I picked up and am keeping here for the moment
nova uid and password : admin / crowbar
Deployed nova/images instance path - on nodes :
/var/lib/libvirt
sudo nova-manage service list
mysql -u<user> -p<password>
Log entries of instance deploy and destroy.
2012-01-13 09:02:57,736 INFO nova.virt.libvirt_conn [-] instance instance-00000007: deleting instance files /var/lib/nova/instances/instance-00000007
2012-01-13 09:02:58,462 INFO nova.virt.libvirt_conn [-] Instance instance-00000007 destroyed successfully.
Since I last messed around a new version of crowbar (1.2) had been released so that was used. A nice addition is rather than research/guess the OpenStack barclamp install order they are now nicely arranged on the Crowbar admin panel.
I also came to the conclusion that my networking 'issues' were a result of both me trying to install on vSphere, which has no support for native VLAN's and by complicating things by trying to shoe horn the networking into my corp range.
So I build a little lab which consisted of anything not nailed to a desk, I could find lying around, or that no-body would notice missing.
Switch - vlans created with SVI ports as needed for default gateways and all ports 802.1q to the nodes.
Crowbar install went fine using Pt1 to fill in the missing details.
The Swift install failed until I added some more storage (those cutting edge USB disks on top) and restarted the Crowbar install from scratch.
My first attempt at building a instance failed. The instance remained in status of 'building'.
To debug
Use the Nova console to check which node the image has been deployed to and then grep the local /var/log/nova/nova-compute.log for ERROR.
2012-01-13 02:52:59,933 ERROR nova.compute.manager [-] Instance '5' failed to spawn. Is virtualization enabled in the BIOS? Details: internal error no supported architecture for os type 'hvm'
This is a result of the crappy hardware I'm using so I changed the nova barclamp to 'qemu' and bingo, deployed instance - well, sometimes. It seems pretty hit and miss, which I can't yet trace. I suspect I could still be suffering from the Heath Robinson lab I've put together. Pt.3 :)
Some useful bits I picked up and am keeping here for the moment
nova uid and password : admin / crowbar
Deployed nova/images instance path - on nodes :
/var/lib/libvirt
sudo nova-manage service list
mysql -u<user> -p<password>
Log entries of instance deploy and destroy.
2012-01-13 09:02:57,736 INFO nova.virt.libvirt_conn [-] instance instance-00000007: deleting instance files /var/lib/nova/instances/instance-00000007
2012-01-13 09:02:58,462 INFO nova.virt.libvirt_conn [-] Instance instance-00000007 destroyed successfully.
2012-01-13 09:07:29,306 INFO nova.virt.libvirt_conn [-] Instance instance-00000009 spawned successfully.
Monday, 6 February 2012
Windows Killed my MBR
Having recently acquired a SSD and some more geebees I re-installed Ubuntu x64. The SSD already had a OEM Windows partition and I (foolishly) decided to just resize it 50/50 and put Linux on the back end. With a mind to come back and build the Windows OS as a dual boot.
Well this morning I went back, built the Windows OS and then fried my GRUB boot loader. Bye bye Ubuntu, insert sad face.
Recovering it wasn't so bad although as usual there is LOTS of conflicting documents and ways to get yourself out of this mess. This method worked for me :
Caveat : The first time I tried to boot Windows it complained of a missing loaded. The second time it worked fine .. I never got to the bottom of that.
Moral of this story is install Windows first and Ubuntu second. Ubuntu is a clever OS and Windows is not.
Well this morning I went back, built the Windows OS and then fried my GRUB boot loader. Bye bye Ubuntu, insert sad face.
Recovering it wasn't so bad although as usual there is LOTS of conflicting documents and ways to get yourself out of this mess. This method worked for me :
- Boot Ubuntu DVD in live mode.
- Jump into a terminal and switch to sudo -i and run parted -l to establish which partition you need to rebuild the MBR for.
~$ parted -l
Model: ATA SAMSUNG SSD PM83 (scsi)
Disk /dev/sda: 128GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 106MB 105MB primary ntfs boot
2 106MB 45.1GB 45.0GB primary ntfs
3 45.1GB 128GB 82.9GB extended
5 45.1GB 120GB 74.4GB logical ext4 <---- Ubuntu Install
6 120GB 128GB 8453MB logical linux-swap(v1)
- mount the partition
mount /dev/<partition> /destination
eg : mount /dev/sda5 /media/sda5
- run
grub-install --root-directory=<mount> <disk>
eg : grub-install --root-directory=/media/sda5 /dev/sdaAssuming you get a success message your good to go. Reboot and you should have Linux back and in my case Windows 7 also now existed on the GRUB loader.
Caveat : The first time I tried to boot Windows it complained of a missing loaded. The second time it worked fine .. I never got to the bottom of that.
Moral of this story is install Windows first and Ubuntu second. Ubuntu is a clever OS and Windows is not.
Sunday, 8 January 2012
New Year, new look
I've been wanting to revamp my website (all two pages of them) and blog design for a while now. I wanted a cut down basic look which I'd seen on other sites. I'm not shy, and 'imitation is the sincerest form of flattery' right !?
Most of the other sites I'd seen sat on Wordpress. I toyed with this and even tested the theory using an AWS cloudformation Wordpress stack. Which incidentally worked really well ! But I've committed to blogger so wasn't going to going to switch that part and learning another platform just for a few pages seemed excessive. Although in hindsight probably less than the path I took.
I started to mess around and build up some html/css. One of my targets was to get a functioning menu rather than the multiple image hack I'd put together before. Reason : I'd stand more of a chance of getting the Google search sub-headings. Thinking about it I don't really know 'why' I want them, but that's now irrelevant.
I found this http://blog.vogtjosh.com/post/2990652547/css3-animated-drop-down-menu which has a really nice looking transition menu - well, I say nice, nice on SOME browsers. Basic on others (looking at you Microsoft).
De-constructing the CSS to get the look I wanted was not quick. A steep learning curve which went back and forth as the smallest change broke the look in different browsers.
I applied the same fade look to the social icons, again, that took ages and often meant going back to square one. Mostly due to my crap web skills but also because of very different behaviour across browsers and because I didn't do any revision control.
Persistence paid off and its mostly finished with the main page, picture gallery and blog all with a similar look. There are a few bugs which I need to look at and I need to clean up and better annotate the CSS. But mostly I'm happy with the result, especially in Chrome, IE - pah, who would use that as a browser now anyway. And I also learnt some new skills which is always nice.
Most of the other sites I'd seen sat on Wordpress. I toyed with this and even tested the theory using an AWS cloudformation Wordpress stack. Which incidentally worked really well ! But I've committed to blogger so wasn't going to going to switch that part and learning another platform just for a few pages seemed excessive. Although in hindsight probably less than the path I took.
I started to mess around and build up some html/css. One of my targets was to get a functioning menu rather than the multiple image hack I'd put together before. Reason : I'd stand more of a chance of getting the Google search sub-headings. Thinking about it I don't really know 'why' I want them, but that's now irrelevant.
I found this http://blog.vogtjosh.com/post/2990652547/css3-animated-drop-down-menu which has a really nice looking transition menu - well, I say nice, nice on SOME browsers. Basic on others (looking at you Microsoft).
De-constructing the CSS to get the look I wanted was not quick. A steep learning curve which went back and forth as the smallest change broke the look in different browsers.
I applied the same fade look to the social icons, again, that took ages and often meant going back to square one. Mostly due to my crap web skills but also because of very different behaviour across browsers and because I didn't do any revision control.
Persistence paid off and its mostly finished with the main page, picture gallery and blog all with a similar look. There are a few bugs which I need to look at and I need to clean up and better annotate the CSS. But mostly I'm happy with the result, especially in Chrome, IE - pah, who would use that as a browser now anyway. And I also learnt some new skills which is always nice.
Thursday, 29 December 2011
Blogger, double line spacing
This has been bugging the crap out of me for ages so I finally sat down this evening and worked it out. Or at least I worked a work-around out !
When creating posts blogger would kindly add </br> tags into the html. </br> is a HTML line break. So when writing a post with CR to create a single line space for a paragraph blogger would kindly add a br tags all over the place, and so you end up with double line spacing.
Note:Changing the line break options made bugger all difference.
I had my default blogger font as Trebuchet but in the WYSIWIG blogger editor the font would show as Times Roman. Each blog post I created I'd select all and use the font drop down to show me the correct font.
Bingo ! .. its at that point blogger gets all crazy with the br tags.
What I hadn't realised is if you leave the editor font alone and look at the preview then you get the default font you've set in the blog template design. And if you don't change the font in the editor then blogger never inserts the </br> tag.
I am happy now. No more switching to HTML view and deleting tags ..
When creating posts blogger would kindly add </br> tags into the html. </br> is a HTML line break. So when writing a post with CR to create a single line space for a paragraph blogger would kindly add a br tags all over the place, and so you end up with double line spacing.
Note:Changing the line break options made bugger all difference.
I had my default blogger font as Trebuchet but in the WYSIWIG blogger editor the font would show as Times Roman. Each blog post I created I'd select all and use the font drop down to show me the correct font.
Bingo ! .. its at that point blogger gets all crazy with the br tags.
What I hadn't realised is if you leave the editor font alone and look at the preview then you get the default font you've set in the blog template design. And if you don't change the font in the editor then blogger never inserts the </br> tag.
I am happy now. No more switching to HTML view and deleting tags ..
Labels:
Blog
VLC, Windows 7 to Basic Mode & Black screen
Debugging an audio issue I have on XBMC and dropped to trusty VLC. Windows (7 x32) flicked into basic mode and VLC only played the audio.
A quick google showed a few hits with the same issue, but no resolution. Check video drivers, update VLC etc. All rather generic.
It also wasn't version specific (I horde older versions) and they all behaved in the same way.
No amount of preference changes resolved it.
Fix : Shut some apps, in my case Chrome was hogging all the RAM and I was >70% used. Releasing some system resources fixed VLC. So not directly a VLC issue.
Hope that helps some people. The problem seems to have been around a while ..
A quick google showed a few hits with the same issue, but no resolution. Check video drivers, update VLC etc. All rather generic.
It also wasn't version specific (I horde older versions) and they all behaved in the same way.
No amount of preference changes resolved it.
Fix : Shut some apps, in my case Chrome was hogging all the RAM and I was >70% used. Releasing some system resources fixed VLC. So not directly a VLC issue.
Hope that helps some people. The problem seems to have been around a while ..
Labels:
VLC
Wednesday, 14 December 2011
Installing flash on Backtrack 5
Backtrack comes with flash installed, but it doesn't work. Believe it or not Nessus requires flash ! A security scanner that requires flash, oh the irony ..
Following the instructions provided by BT didn't work for me so I'm reproducing the 32bit steps which did :
Grab the correct/latest flash install from adobe.com :
UPDATE : This process also works for Chromium
Following the instructions provided by BT didn't work for me so I'm reproducing the 32bit steps which did :
Grab the correct/latest flash install from adobe.com :
install_flash_player_11_linux.i386.tar.gzRun the following commands :
Remove BT installed files
apt-get purge flashplugin-nonfree flashplugin-installer gnash gnash-common mozilla-plugin-gnash swfdec-mozillarm -f /usr/lib/firefox/plugins/*flash*
rm -f /usr/lib/firefox-addons/plugins/*flash*rm -f /usr/lib/mozilla/plugins/*flash*
rm -f ~/.mozilla/plugins/*flash*sorm -rfd /usr/lib/nspluginwrapper
Install downloaded version
tar xvfz install_flash_player_11_linux.i386.tar.gz
mkdir ~/.mozilla/plugins
mv -f libflashplayer.so ~/.mozilla/plugins/Test within the browser, firefox about:plugins
UPDATE : This process also works for Chromium
Labels:
Backtrack
Wednesday, 30 November 2011
Crowbar & OpenStack Install Part 1
A few weeks ago I decided to investigate OpenStack. The core components are a collaboration and resulting love child of NASA and RackSpace and is an Open Source solution to build a blaah blaah cloud. It seems to be gaining good traction and is often mentioned in the same breath as VMware. In fact it appears everyone not in VMwares back pocket is jumping on the bandwagon and so seemed worthy of a look.
This is still a moving target for me, its not 'up' and has certainly not been painless and smooth, but I'm writing up Pt.1 as much for me as for the blogosphere.
OpenStack is, well, just that, a stack of different products which together produce an end solution. In my eyes and how I'd describe it to 'management' is its comparable to LAMP. Linux, Apache, MySQL and PHP.
I won't go into the details you can read yourself - http://en.wikipedia.org/wiki/OpenStack
I started by reading the install docs and attempting to install each piece individually and then I guess hook them all together.
This didn't go well and a twitter reply from Matthew McClean suggested I take look at Dells Crowbar.
The rest of this post will be the process, troubleshooting and useful links I've come across as I attempt to install Crowbar on my journey to OpenStack nirvana ..
Crowbar background
https://github.com/dellcloudedge/crowbar/wiki
http://robhirschfeld.com/2011/03/14/how-openstack-installer-works/
Resources
4 VMware guests - Each has two NICS each on a vlan, eth0 vlan 66 and eth1 99. eth0 will be re-configured from the default so its available on my corp LAN.
Installation
The documented install procedure - https://github.com/dellcloudedge/crowbar/wiki/Install-crowbar
If only it were that simple.
Theres a nice YouTube video of building the environment which I started to follow :
Another little find went into some further details - http://kb.dtosolutions.com/wiki/Deploying_a_small_OpenStack_test_environment_using_Crowbar
From these I eventually cooked up the following install method which seems to be more successful than anything else so far.
Assuming you have the admin node built and three green nodes.
Throughout all of this I had a tail -f running to the admin log :
By all accounts I should now have a working environment. I don't, everything appears to be installed but the learning curve continues.
Nova dashboard is not working which I've traced to a routing issue on node 2. It has the admin interface up but the default gateway is using one of the networks I didn't change at the install time.
Nagios is not working. Watching the production.log during the barclamp installs showed Nagios errors but I'm ignoring them/that for now.
Troubleshooting / Hints 'n' tips :
Barclamps sometimes report failure to deploy but repeating the same task will be successful.
If your running your POC on hypervisors or something else that has the technology make good use of snapshots.
This is still a moving target for me, its not 'up' and has certainly not been painless and smooth, but I'm writing up Pt.1 as much for me as for the blogosphere.
OpenStack is, well, just that, a stack of different products which together produce an end solution. In my eyes and how I'd describe it to 'management' is its comparable to LAMP. Linux, Apache, MySQL and PHP.
I won't go into the details you can read yourself - http://en.wikipedia.org/wiki/OpenStack
I started by reading the install docs and attempting to install each piece individually and then I guess hook them all together.
This didn't go well and a twitter reply from Matthew McClean suggested I take look at Dells Crowbar.
The rest of this post will be the process, troubleshooting and useful links I've come across as I attempt to install Crowbar on my journey to OpenStack nirvana ..
Crowbar background
https://github.com/dellcloudedge/crowbar/wiki
http://robhirschfeld.com/2011/03/14/how-openstack-installer-works/
Resources
4 VMware guests - Each has two NICS each on a vlan, eth0 vlan 66 and eth1 99. eth0 will be re-configured from the default so its available on my corp LAN.
- Admin Node - 2GB RAM, 20GB disk
- Node 1 - 1GB RAM, 15GB disk
- Node 2 - 1GB RAM, 8GB disk
- Node 3 - 1GB RAM, 8GB disk
Installation
The documented install procedure - https://github.com/dellcloudedge/crowbar/wiki/Install-crowbar
If only it were that simple.
- Build or grab the correct ISO - in this case v1.1 http://crowbar.zehicle.com/openstack111111.iso
NB//Crowbar.iso is a base install, it has none of the OpenStack components.
- Boot your first server on the ISO
- If the default networking works for you then great. Otherwise edit network BEFORE installing. In my case I changed the Admin network settings.
- Run install - https://github.com/dellcloudedge/crowbar/wiki/Install-crowbar
This will build, pretty seamlessly, your 'admin' node which is then used to build the other components.Here things went a little squiffy.
Theres a nice YouTube video of building the environment which I started to follow :
But with little success. The first barclamp (name assigned to the Crowbar packages, which actually in turn directly represent Chef cookbooks) which the video has you assign is Glance, which for me failed. Attempting to resolve this I joined the Crowbar lists and co-coincidently someone else asked about the order to install applications and had another link :
MySQLVery different to the order on the video ! - But its still worthwhile watching.
Keystone
Nova Dashboard
Glance
Nova
Another little find went into some further details - http://kb.dtosolutions.com/wiki/Deploying_a_small_OpenStack_test_environment_using_Crowbar
From these I eventually cooked up the following install method which seems to be more successful than anything else so far.
Due to a bug in the dashboard when removing or adding nodes I found you had to create all your nodes upfront before you start to apply barclamps. Create 3 nodes, power them all on so they run the discovery image and become available in the Crowbar console (unassigned, yellow).
Click bulk edit and assign all three. They will install the OS, reboot a few times and then go 'assigned' in the console with a green icon.
Assuming you have the admin node built and three green nodes.
- Build MySQL proposal and assign server to node 1 and client to node 2 and 3
- Build Keystone and assign to node 1
- Nova Dashboard to node 1
- Glance to node 1
- Nova.
"libvirt_type": "kvm", -> "libvirt_type": "qemu",
And assign Controller to node 1 and then compute to 2 and 3.
Throughout all of this I had a tail -f running to the admin log :
/opt/dell/crowbar_framework/log/production.log
By all accounts I should now have a working environment. I don't, everything appears to be installed but the learning curve continues.
Nova dashboard is not working which I've traced to a routing issue on node 2. It has the admin interface up but the default gateway is using one of the networks I didn't change at the install time.
Nagios is not working. Watching the production.log during the barclamp installs showed Nagios errors but I'm ignoring them/that for now.
Troubleshooting / Hints 'n' tips :
Barclamps sometimes report failure to deploy but repeating the same task will be successful.
If your running your POC on hypervisors or something else that has the technology make good use of snapshots.
Wednesday, 23 November 2011
Empathy new IM notifications / window
Part of my continued insistence on using Linux rather than Windows has meant using Pidgin as my IM client for corporate Sametime and Google chat.
Not being 100% happy with it I switched to Empathy. Much cleaner interface and generally more happy.
But, new IM conversations appeared on the notifications bar (Gnome3) rather than launching a new window. Annoying and very easily missed.
Nothing obvious in the preferences. Clutched at straws by disabling (Preferences | General) 'Display incoming events in the notification area' and bingo.
New tabs/windows open on a new IM - happy (for now)
Not being 100% happy with it I switched to Empathy. Much cleaner interface and generally more happy.
But, new IM conversations appeared on the notifications bar (Gnome3) rather than launching a new window. Annoying and very easily missed.
Nothing obvious in the preferences. Clutched at straws by disabling (Preferences | General) 'Display incoming events in the notification area' and bingo.
New tabs/windows open on a new IM - happy (for now)
Monday, 24 October 2011
OCD, me !?
A conversation in the office on how to lace shoes, something I have very strong feelings about, resulted in me making a list :
- Shoe laces must be over not under and be symmetrical.
- If a shirt has button down collars - use them !
- Shirt/T-shirts are not allowed to have the top button done up. Unless a tie is involved.
- Under no circumstance will anyone touch my screens with fingers.
- Toilet rolls, over not under.
- Cables will not be crossed in my data centre.
- Wooden cutlery, simply not allowed.
- Rattles in car. Must be found.
- Plastic film on remote controls/electronic devices - not allowed and will be removed ASAP.
Labels:
OCD
Tuesday, 4 October 2011
Powershell file & directory purge script
I've been wanting to learn something new for a while and since @darrenparkinson comments on my DNS update batch file I thought I'd have a look at Powershell, aka POSH (I like that).
Started by diligently reading each chapter of 'Mastering Powershell', Dr. Tobias Weltner. I say each chapter, my attention levels waned after about chapter, erm 1.
Plan B.
I already have a few scripts (batch) running in the office so picked on one of those and started trying to write a replacement. In this case a purge files after # days which we run on a fileshare.
Dive in, used the book for reference and of course the worlds saviour our good friend google.
Lots of reading, lots of searching, lots of head scratching, lots of trying others examples and lots of grumpiness resulted in below.
# // ----------------------------------------------------------------------------------//
Started by diligently reading each chapter of 'Mastering Powershell', Dr. Tobias Weltner. I say each chapter, my attention levels waned after about chapter, erm 1.
Plan B.
I already have a few scripts (batch) running in the office so picked on one of those and started trying to write a replacement. In this case a purge files after # days which we run on a fileshare.
Dive in, used the book for reference and of course the worlds saviour our good friend google.
Lots of reading, lots of searching, lots of head scratching, lots of trying others examples and lots of grumpiness resulted in below.
# // ----------------------------------------------------------------------------------//
# // This script will delete files and directories based on path and days set below //
# // Errors are written to the Application event log and an email summary sent. //
# // //
#
# // Revision history :
# // Created : 03/10/2011 - Paul Regan //
# // ----------------------------------------------------------------------------------//
// READ THESE TWO SECTIONS !!!
//
// 1. Before you can start writting to system logs you have to register the source - only once. Not particulary well documented so left here
// for reference :
// New-EventLog -Logname "<name>" -Source "<a name you can reference>"
// eg
// new-EventLog -Logname "Application" -Source "POSH:Script"
// 2. Depending on your local security you may need to set script execution policy.
// http://technet.microsoft.com/en-us/library/ee176961.aspx
//
// set-executionpolicy <level>
// eg
// set-executionpolicy unrestricted
# // *********** Set your variables here !
# //
# // -Days ## is the period you want to retain
$span = new-timespan -Days 30
# // Working directory
$SourcePath = "s:\temporary"
# // Set logging variables
$logname = "s-temp-delete-$todaynumerical.log"
$logpath = "c:\scripts\logs"
# // Set eMail variables
$subject = "POSH Script : Files and directories deleted from $sourcepath on $todaynumerical"
$fromAddress = "posh@domain.co.uk"
$smtpServerName = "smtpserver.domain.co.uk"
$toAddress = "people@domain.co.uk"
# // Send attachment or results in the body of the email [true/false]
$emailattach = "false"
# // System variable - DONT CHANGE unless you know what you are doing (not that I do)
$today = get-date
$todaynumerical = get-date -format "yyyy-d-M"
$CutoffTime = $today-$span
# // Creates a function with all the param needed for the commane Write-EventLog
function WriteEventLog ($message, $entrytype)
{
Write-EventLog -Logname "Application" -Source "FilesScript" -Message $message -EntryType $entrytype -EventID 4001
}
# // Set the two variables
$message = "PS Log file message"
$entrytype = "Information"
# // I think this writes the first message to the Windows event log defined above, gives you a 'start of process' point to find in a busy log.
WriteEventLog $message $entrytype
# // Builds a file list (PS can accept/apply multiple output messages to the same variable.
# // Where-Object {$_.PSIsContainer -eq $False = Only files, directories would return TRUE
$files = get-childitem -recurse $SourcePath | Where-Object {$_.PSIsContainer -eq $False}
# // empty arrays to hold file lists
$removedFiles = @()
$skippedFiles = @()
$failedactionfiles =@()
# // Start the for loop to read $files and determin the action based on the modified date
foreach ($file in $files)
{
remove-item -ev a -path $file.fullname -recurse -force -whatif | Where-Object {$_.lastwritetime -le $CutoffTime}
<#
#####################################################################################################################
if ($file.lastwritetime -gt $CutoffTime)
{
$skippedFiles += $file.fullname
# $message = "Not moving newer: " -join $file.fullname
# $entrytype = "Information"
# // I don't really want to write files not deleted/moved into the EventLog
# WriteEventLog $message $entrytype
}
else
{
# // Build the newname variable using destination path, old file name and todays date
# $newname = $DestPath + "\" + $file.name + "-" + $file.lastwritetime.tostring("yyyy-MMM-dd")
# // -ev = ErrorVariable which the output from is being set to 'a'
# // $file.fullname pulls the full path from the object (file)
#move-item -ev a -path $file.fullname -destination $newname
remove-item -ev a -path $file.fullname -recurse -force -whatif
# // From the variable $a .count the number of entries, entries against $a would mean an error
#####################################################################################################################
#>
# // Are there any errors ?
if ($a.count -gt 0)
{
# // Creates a $message variable based on $a (output from Remove-Item) and $file (The file we're working on)
# // remote-item generates a nice clean message. move-item does-not, both message options below.
#$message = $file.fullname + " - " + $a[0].tostring()
$message = $a[0].tostring()
# // Writes the error to the event log defined above
$type = "Error"
WriteEventLog $message $type
# // Also add the message to the $failedactionfiles array, which we'll use later ..
$failedactionfiles += $message
}
else
{
$removedFiles += $file.fullname
}
}
# // I have now removed all files older than $span and will start removing all EMPTY directories that match the same cut off time
# // Build a list of home directories from $sourcepath and apply to $rootdirs
$rootdirs = get-childitem $sourcepath | Where-Object {$_.PSIsContainer -eq $True}
<# // This function is not actually being used but I am proud of it, my first, so its staying here for now.
function getemptydirs ($source)
{
# // Command from http://technet.microsoft.com/en-us/library/ff730953.aspx
# // (Get-ChildItem c:\scripts\working\live -recurse | Where-Object {$_.PSIsContainer -eq $True}) | Where-Object {$_.GetFiles().Count -eq 0} | Select-Object FullName,root,parent
(Get-ChildItem $source -recurse | Where-Object {$_.PSIsContainer -eq $True}) | Where-Object {$_.GetFiles().Count -eq 0} | Select-Object FullName
}
# // Get-ChildItem c:\scripts\working\live\paul | get-member << to display available properties
# // (Get-itemproperty -path c:\scripts\working\live\paul).lastwritetime << to display lastwrite time property
# // Get-ChildItem c:\scripts\working\live -recurse | Where {$_.psIsContainer -eq $true} | get-member -membertype property
# // Get-ChildItem c:\scripts\working\live -recurse | Where {$_.psIsContainer -eq $true} | get-itemproperty -name "Parent"
#>
# // Create some empty arrays and default options to be used
$emptydirs = @()
$removeddirs = @()
$message = "PS Log file message"
$entrytype = "Information"
# // Loop through $rootdirs and create an array containing all directories with zero file count and older than # days (set above)
foreach ($directory in $rootdirs)
{
$emptydirs += (Get-ChildItem $directory.fullname -recurse | Where-Object {$_.PSIsContainer -eq $True}) | Where-Object {$_.lastwritetime -le $CutoffTime} | Where-Object {$_.GetFiles().Count -eq 0}
}
# // Now delete the list we just made
# // First compare the count of $rootdirs & $emptydirs. If they are the same it indicates there are no directories to delete.
if ($emptydirs.count -eq $rootdirs.count) {
echo "Zero to delete"}
else {
# // Loop through $emptydirs removing each directory
# // NB// If the root dir is actually empty, ie has no dirs that match the remove filter then it will generate a null error if you watch the script live.
foreach ($emptydir in $emptydirs) {
remove-item -errorvariable err -path $emptydir.fullname -recurse -force -whatif
if ($err.count -gt 0) {
# // Creates a $message variable based on $error (output from remove-item)
$message = $err[0].tostring()
# // Writes the error to the event log defined above
#$type = "Error"
#WriteEventLog $message $type
# // Also add the message to the $failedactiondir array, which we'll use later ..
$failedactiondir += $message }
else {
$removeddirs += $emptydir.fullname}
}
}
# // Create the log files
# // Moosh both the results arrays into a single with output to a txt file
$attachments = $removedfiles+$removeddirs | out-file $logpath\$logname
# // Just to keep the email clean I'll check for empty file actions and set a cute little message
if ($failedactionfiles.count -eq 0) {$failedactionfiles = "No file remove failures"}
if ($removedfiles.count -eq 0) {$removedfiles = "No files removed"}
if ($skippedfiles.count -eq 0) {$skippedfiles = "No files skipped"}
if ($removeddirs.count -eq 0) {$removeddirs = "No directories removed"}
# // Construct the email message
$bodyText = @()
$bodyText += "Failed to remove: "
$bodyText += $failedactionfiles
$bodyText += "----------------------------------------------------"
$bodyText += "Removed files: "
$bodyText += $removedFiles
$bodyText += "----------------------------------------------------"
$bodyText += "Skipped files: "
$bodyText += $skippedFiles
$bodyText += "----------------------------------------------------"
$bodyText += "Removed directories: "
$bodyText += $removeddirs
# // The text from the array $bodyText when sent via email comes out as a single line. | Out-String deals with that.
# // http://social.technet.microsoft.com/Forums/en-US/winserverpowershell/thread/e0213b00-331f-4588-be90-afd0ce5ed50b/
$bodyText = $bodyText | Out-String
$bodyString = @"
$bodyText
"@
if ($emailattach -eq "false") {
# // Send email with results in the body
send-mailmessage -To $toAddress -Subject $subject -From $fromAddress -Body $bodyString -SmtpServer $smtpServerName
}
else {
# // Send email with attachment log file
send-mailmessage -To $toAddress -Subject $subject -From $fromAddress -attachments "$logpath\$logname" -SmtpServer $smtpServerName
}
So my journey is far from over, in fact its just started and I'll re-visit this at some point as its pretty scrappy and no doubt horribly inefficient, but it works and I even managed to improve on the previous script as before empty directories got left behind and thats now addressed with this so I'm happy(ish). I at least now have a much better understanding of the potential.
I'm also resigned that I'll never be a developer and I can certainly relate to this girl ..
Replace Pi with many POSH cmdlets and statements I've attempted to understand over the past month or so .....
I'm also resigned that I'll never be a developer and I can certainly relate to this girl ..
Replace Pi with many POSH cmdlets and statements I've attempted to understand over the past month or so .....
Labels:
Scripting
What is an Object?
What is an Object?
An “Object” is something we can gather information from and/or perform an action upon. An object, in PowerShell, consists of properties (information we can gather) and methods (actions we can perform).
As an example, let’s look at a “light bulb.” The object should be obvious, it’s a light bulb. The properties of a light bulb could be color, wattage, and type (florescent, incandescent, or halogen). Methods are the actions we can perform on the object such as; turn on the light bulb and turn off the light bulb.
Labels:
Powershell
Wednesday, 7 September 2011
XBMC Addons (finding the bloody things)
After wasting too much time trying to find an addon I'd just installed.
Go to the type of media, eg video, and navigate up !
Go to the type of media, eg video, and navigate up !
Labels:
XBMC
Wednesday, 31 August 2011
Visio Vertices
Because literately everytime I go to do this I forget, its my Visio black hole ..
Vertices appear when you select the Line or Pencil tool !
Labels:
Microsoft Visio
Tuesday, 30 August 2011
Humax IR problem, BlueEye to the rescue !
Replacing my Sky service with a Humax receiver has only been a good thing but the IR 'issue' was bugging me. It just wasn't up to the job in the way my room is laid out.
I settled on an IR extended from BlueEye, ordered, arrived, excelled service.
I tested it for a few days and it solved the problem. I made the final install by routing the cable through the case and tucking the IR sender into the back of the IR component on the front facing board. Its a few screws and the front pops off, all pretty simple.
Friday, 19 August 2011
Getting the cute profile picture in Google search results
Looking for something this morning and I noticed peoples picture appearing next to some searches :
Apparently easy. Well the first part was, getting a full sign off on the webmaster snippets testing tool took a fair amount of reading.
![]() |
Some random person |
You need :
Your profile url will be something like : https://plus.google.com/102726977217362443519
NB// the rel="author" is the super important bit.
I already had a Google+ link (top left) on my blog so just added the rel="author"

Next step is to edit your Google profile so that it includes a link to the site you just added the link on.
Running the snippet test tool against this blog gave me a single error :
The pertinent part for me was the class='timestamp-link' section with the class 'updated'. And the part missing from my blogger template.
Edit the design of your template and go into raw edit HTML, enable the widget check box and then search for the text 'timestamp-link'.
In my template there were two entries. Add 'updated' to the class. Save and re-run the snippet test, you should be good.
- Website, Blog, or I think pretty much anything online would work as long as you can edit the code
- Google profile
Your going to create a link between those two things which Google magic uses to look you up in searches that hit your content.
On your site/blog create a link to your Google profile :
NB// the rel="author" is the super important bit.
I already had a Google+ link (top left) on my blog so just added the rel="author"

Next step is to edit your Google profile so that it includes a link to the site you just added the link on.
Running the snippet test tool against this blog gave me a single error :
Warning: Missing required field "updated".4 hours later and much reading I came across this which talks about enabling your blog for hatom, part of the microformats initiative - and way out of the scope for this post.
The pertinent part for me was the class='timestamp-link' section with the class 'updated'. And the part missing from my blogger template.
Edit the design of your template and go into raw edit HTML, enable the widget check box and then search for the text 'timestamp-link'.
In my template there were two entries. Add 'updated' to the class. Save and re-run the snippet test, you should be good.
<a class='timestamp-link updated' expr:href='data:post.url' rel='bookmark' title='permanent link'><abbr class='published' expr:title='data:post.timestampISO8601'><data:post.timestamp/></abbr></a>So now you wait <insert Google time> until they get around to crawling the site and updated the index.
Subscribe to:
Posts (Atom)