Migrate from RPM nginx to source compiled nginx

22 August 2013 13:10

If you’ve installed nginx via rpm/yum you’ll be a bit stuck if you then decide you want to add some modules. In order to add modules, or, use something else you need to compile from source.

Below are the steps I took to get this going. This is on CentOS 6.4.

You’ll need to map things to where they were if you want to keep the exact same locations.

for i in nginx.conf nginx.pid client_body_temp proxy_temp fastcgi_temp uwsgi_temp scgi_temp; do echo "$i:"; locate -b $i; done

You’ll need some pre-requisites.

sudo yum -y install pcre pcre-devel libxml2 libxml2-devel curl curl-devel httpd-devel unzip openssl openssl-devel;
wget http://nginx.org/download/nginx-1.4.2.tar.gz;
wget https://github.com/pagespeed/ngx_pagespeed/archive/release-1.6.29.5-beta.zip;
wget https://dl.google.com/dl/page-speed/psol/1.6.29.5.tar.gz;
tar xfz nginx-1.4.2.tar.gz;
tar xfz 1.6.29.5.tar.gz;
unzip release-1.6.29.5-beta;
mv psol ngx_pagespeed-release-1.6.29.5-beta;

Once you’re all unpacked you can get building.

cd nginx-1.4.2;
./configure \
--prefix=/usr \
--conf-path=/etc/nginx/nginx.conf \
--pid-path=/var/run/nginx.pid \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/acces.log \
--http-client-body-temp-path=/var/cache/nginx/client_body_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--with-http_gzip_static_module \
--with-http_addition_module \
--with-http_stub_status_module \
--add-module=../ngx_pagespeed-release-1.6.29.5-beta; 
make;
sudo make install;

Now you’ve installed it, replacing the binary in place without dropping traffic.

sudo kill -s USR2 `cat /var/run/nginx.pid`;

Now, I did try compiling this all with mod security 2.7.5 but get segfaults all over. Needs more investigation to figure out why!

Filed: General // No Comments

Move objects Rackspace Cloud Files

20 August 2013 10:51

I had a need to move a bunch of files to a new container in Rackspace’s Cloud Files. Using the handy API Python Bindings (pyrax) this is pretty easy.

#!/usr/bin/python2.7
import os
import sys
import pyrax
import pyrax.exceptions as exc
import pyrax.utils as utils
creds_file = os.path.expanduser(".rackspace_cloud_credentials")
pyrax.set_setting("identity_type", "rackspace")
pyrax.set_setting("region", "LON")
pyrax.set_credential_file(creds_file)
cf = pyrax.cloudfiles
oldcont = cf.get_container("old_container")
newcont = cf.get_container("new_container")
objects = oldcont.get_objects()
counter = 0
for obj in objects:
 cf.move_object(oldcont, obj.name, newcont)
 counter += 1
 print "[%d] %s moved from %s to %s" % (counter, obj.name, oldcont.name, newcont.name)

The credentials file, if you’re wondering, looks like this:

[rackspace_cloud]
username = USERNAME
api_key = KEY

The follow up is to port this to ruby/fog/rumm once I figure out ruby and get time :)

Filed: General // No Comments

Jekyll – stack level too deep (SystemStackError)

20 May 2013 14:57

The latest version of Jekyll has come out and I’ve been playing at converting my WordPress blog again. Messing around with ruby versions and arguments (not being a rubyist!) keep getting an odd error.

Using 2.0.0-dev:

$ ruby -r './wordpress.rb' \
  -e 'JekyllImport::WordPress.process({dbname:"db",user:"user"})'
/path/to/wordpress.rb:260: stack level too deep (SystemStackError)

Using 1.9.3-p327, the last character on line 260 had been causing pain:

$ ruby -r './wordpress.rb' \
  -e 'JekyllImport::WordPress.process({dbname:"db",user:"user"})'
/path/to/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': /path/to/wordpress.rb:260: invalid multibyte char (US-ASCII) (SyntaxError)
/path/to/wordpress.rb:260: syntax error, unexpected $end
 }.delete_if { |k,v| v.nil? || v == '' }.to_yaml¬

This was almost certainly introduced during copy and pasting between vim windows. C&P death! Interesting fact those is ruby 2.0.0-dev hid that error, well, hid the detail of it anyway.

I was originally Paul Stamatiou‘s version, but, am now trying the version from the beta gem.

Also note that I built the gem manually before I did all this:

git clone git://github.com/jekyll/jekyll-import.git
gem build jekyll-import.gemspec
gem install ./jekyll-import-0.1.0.beta1.gem

I did have some other hacks from icebreaker and myself in the last version, so, I’ll get those in and hopefully publish it.

Filed: General // 1 Comment

CDN Debugging

20 May 2013 12:28

Working with a CDN can be a whole world of pain sometimes, especially when you’re trying to figure out what the heck is going on.

Firstly, CDN’s quite often will have a pair of POP’s. Someone like LLNW will return you this:

$ host media.limelight.com
media.limelight.com is an alias for llnw.vo.llnwd.net.
llnw.vo.llnwd.net has address 87.248.213.101
llnw.vo.llnwd.net has address 178.79.207.22
llnw.vo.llnwd.net has IPv6 address 2a02:3d0:600:15:230:48ff:feda:d0c0
llnw.vo.llnwd.net has IPv6 address 2a02:3d0:600:212:225:90ff:fe4a:ef16

Each POP could return you different content. Whilst not recommended you could target a POP directly like this:

curl -H 'Host: media.limelight.com' -H 'X-LDebug:2' \
  -I http://87.248.213.101/designimages/limelight-content-delivery.jpg

LimeLight Networks offer some debugging via a header which is shown in the command above, “X-LDebug:2”. If you make a call with it in you’ll see some extra X-Cache information:

$ curl -H 'Host: media.limelight.com' -H 'X-LDebug:2' \
  -I http://87.248.213.101/designimages/limelight-content-delivery.jpg
HTTP/1.1 200 OK
Server: Apache
Cache-Control: max-age=3600
X-Server-Name: la-c1-r3-u30-b1
Content-Type: image/jpeg
Via: 1.1 la-c1-r2-u20-b2:3128 (squid)
X-Cache: MISS from cds1020.iad.llnw.net
X-Cache: MISS from cds822.lga.llnw.net
Age: 308
Date: Mon, 20 May 2013 11:15:05 GMT
Last-Modified: Thu, 08 Mar 2012 22:18:12 GMT
Expires: Mon, 20 May 2013 12:09:57 GMT
X-Cache: HIT from cds163.lon.llnw.net
Content-Length: 5486
X-Cache: MISS from cds761.lon.llnw.net
Connection: keep-alive

Akamai are a much larger network and as such can be a bit more complex to find POPs, but, they also offer a lot more in terms of debugging. Most of this is lifted from Mesmor.

Akamai provide an extension for both Internet Explore and Firefox which can be found in the control panel. You can also quickly see the information from an Akamai asset using curl is:

curl -I -H "Pragma: akamai-x-cache-on, akamai-x-cache-remote-on, 
akamai-x-check-cacheable, akamai-x-get-cache-key, akamai-x-get-extracted-values, 
akamai-x-get-nonces, akamai-x-get-ssl-client-session-id, akamai-x-get-true-cache-key, 
akamai-x-serial-no" http://www.website.com/style.css

Some explanation of the results:

X-Cache : Cache State

X-Cache TCP_MEM_HIT from a118-214-191-15 (AkamaiGHost/6.7.0.2-9183648)

a118-214-191-15      : Akamai IP Identifier (The Akamai 
                       server that serves this request)
TCP_HIT              : The object was fresh in cache and 
                       object from disk cache.
TCP_MISS             : The object was not in cache, server 
                       fetched object from origin.
TCP_REFRESH_HIT      : The object was stale in cache and we 
                       successfully refreshed with the origin 
                       on an If-modified-Since request.
TCP_REFRESH_MISS     : Object was stale in cache and refresh 
                       obtained a new object from origin 
                       in response to our IF-Modified-Since request.
TCP_REFRESH_FAIL_HIT : Object was stale in cache and we 
                       failed on refresh (couldn't reach origin) 
                       so we served the stale object.
TCP_IMS_HIT          : IF-Modified-Since request from client 
                       and object was fresh in cache and served.
TCP_NEGATIVE_HIT     : Object previously returned a "not found"
                       (or any other negatively cacheable response)
                       and that cached response was a hit for this
                       new request.
TCP_MEM_HIT          : Object was on disk and in the memory 
                       cache. Server served it without hitting 
                       the disk.
TCP_DENIED           : Denied access to the client for whatever reason.
TCP_COOKIE_DENY      : Denied access on cookie authentication 
                       (if centralized or decentralized authorization 
                       feature is being used in config).

X-Cache-Key : Internal Cache Key

X-Cache-Key /L/1745/15811/3h/www.website.com/common/css/style.css
15811           : Akamai CP Code
3h              : Cache TTL
www.website.com : Origin Server Name

X-Check-Cacheable : Cacheability

This determines if the request object is cache-able as set in your Akamai configuration.

You can read more about CDN testing pre-live and also accessing some other providers headers.

Filed: General // No Comments

ColdFusion Lockdown

20 May 2013 11:56

So been going through various ColdFusion lock down tweeks, restrictions etc. and have a couple of useful links.

First up is the Unofficial Updater. What it’ll do is download the latest, relevant patches for your CF install and get them all in place. It saves a huge amount of time versus trying to do it yourself.

Second up is something off the back of the lock down guide relating to GraphData.cfm. It’s not a real path, it’s something that CF sorts out itself. If you choose to move CFIDE or restrict the path in the webserver this can cause issues. Either you leave it open, or, you can change the path it uses.

Credit to Brandon for the following tweaks to move it about:

{cfmx-root}/lib/neo-graphing.xml
Below changes cfchart engine to generate the image path based on this config.
/CFIDE/GraphData.cfm ==> /images/GraphData.cfm

{cfmx-root}/wwwroot/WEB-INF/web.xml servlet mapping
When a request is handled to /images/Graphdata.cfm the GraphServlet will be 
invoked to find and serve the charts.
/CFIDE/GraphData.cfm ==> /images/GraphData.cfm

Filed: General // No Comments

Multiple “open with” entries on Mac fix

5 March 2013 10:14

If you’re running Mac OS you might have lots of duplicate entries in your “open with” menu. Thanks to a quick Google I found this post which details the fix, which, is nice and simple!

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user

You’ll need to the relaunch /force quit finder or reboot.

Only issue I had was a couple of default “open with” options had reset, but, get info on those types and reset takes no time at all.

Filed: General // No Comments

Star Wars traceroute

15 February 2013 13:55

Kind of old news now, but, I’ll post this anyway.

Take a read of this post. To find out what it’s all about and how it was done. If you don’t understand or can’t be bothered, here’s the output.

$ traceroute 216.81.59.173
traceroute to 216.81.59.173 (216.81.59.173), 64 hops max, 52 byte packets
 1 192.168.50.1 (192.168.50.1) 0.572 ms 0.375 ms 0.364 ms
 2 84.19.53.18 (84.19.53.18) 0.575 ms 0.565 ms 0.527 ms
 3 84.19.53.2 (84.19.53.2) 2.520 ms 2.935 ms 2.440 ms
 4 88.211.81.53 (88.211.81.53) 2.662 ms 2.977 ms 2.623 ms
 5 v207.core1.lon1.he.net (216.66.80.177) 5.594 ms 10.048 ms 13.509 ms
 6 10gigabitethernet2-4.core1.par2.he.net (72.52.92.42) 9.420 ms 12.393 ms
 10gigabitethernet7-4.core1.nyc4.he.net (72.52.92.241) 71.061 ms
 7 10gigabitethernet2-3.core1.ash1.he.net (72.52.92.86) 81.497 ms 76.879 ms 84.098 ms
 8 10gigabitethernet1-2.core1.atl1.he.net (184.105.213.110) 96.704 ms 99.003 ms 99.772 ms
 9 216.66.0.26 (216.66.0.26) 95.045 ms 93.482 ms 103.896 ms
10 10.26.26.102 (10.26.26.102) 129.185 ms 127.454 ms 128.497 ms
11 episode.iv (206.214.251.1) 127.964 ms 130.673 ms 126.375 ms
12 a.new.hope (206.214.251.6) 132.574 ms 135.954 ms 132.619 ms
13 it.is.a.period.of.civil.war (206.214.251.9) 137.614 ms 134.545 ms 132.662 ms
14 rebel.spaceships (206.214.251.14) 136.608 ms 148.727 ms 136.672 ms
15 striking.from.a.hidden.base (206.214.251.17) 138.535 ms 130.344 ms 128.546 ms
16 have.won.their.first.victory (206.214.251.22) 131.485 ms 129.979 ms 128.568 ms
17 against.the.evil.galactic.empire (206.214.251.25) 129.952 ms 132.530 ms 134.179 ms
18 during.the.battle (206.214.251.30) 136.926 ms 137.240 ms 128.703 ms
19 rebel.spies.managed (206.214.251.33) 129.211 ms 129.687 ms 131.459 ms
20 to.steal.secret.plans (206.214.251.38) 137.336 ms 134.963 ms 137.010 ms
21 to.the.empires.ultimate.weapon (206.214.251.41) 137.164 ms 132.548 ms 136.727 ms
22 the.death.star (206.214.251.46) 138.797 ms 135.106 ms 131.307 ms
23 an.armored.space.station (206.214.251.49) 131.542 ms 127.945 ms 127.242 ms
24 with.enough.power.to (206.214.251.54) 132.804 ms 136.714 ms 136.465 ms
25 destroy.an.entire.planet (206.214.251.57) 137.520 ms 131.996 ms 132.086 ms
26 pursued.by.the.empires (206.214.251.62) 127.587 ms 128.695 ms 133.108 ms
27 sinister.agents (206.214.251.65) 132.133 ms 130.615 ms 131.129 ms
28 princess.leia.races.home (206.214.251.70) 136.134 ms 148.275 ms 147.835 ms
29 aboard.her.starship (206.214.251.73) 147.638 ms 141.807 ms 129.552 ms
30 custodian.of.the.stolen.plans (206.214.251.78) 140.750 ms 142.327 ms 150.433 ms
31 that.can.save.her (206.214.251.81) 150.344 ms 145.455 ms 139.564 ms
32 people.and.restore (206.214.251.86) 141.018 ms 133.512 ms 133.397 ms
33 freedom.to.the.galaxy (206.214.251.89) 135.804 ms 130.033 ms 131.854 ms
34 0-----i-------i-----0 (206.214.251.94) 131.347 ms 131.364 ms 136.892 ms
35 0------------------0 (206.214.251.97) 137.222 ms 138.468 ms 135.353 ms
36 0-----------------0 (206.214.251.102) 130.934 ms 131.678 ms 131.364 ms
37 0----------------0 (206.214.251.105) 131.165 ms 128.787 ms 132.123 ms
38 0---------------0 (206.214.251.110) 131.030 ms 130.483 ms 136.405 ms
39 0--------------0 (206.214.251.113) 136.736 ms 137.485 ms 139.646 ms
40 0-------------0 (206.214.251.118) 132.272 ms 129.410 ms 131.636 ms
41 0------------0 (206.214.251.121) 130.995 ms 137.218 ms 136.325 ms
42 0-----------0 (206.214.251.126) 138.280 ms 137.995 ms 138.286 ms
43 0----------0 (206.214.251.129) 136.456 ms 137.882 ms 137.453 ms
44 0---------0 (206.214.251.134) 132.797 ms 131.182 ms 132.678 ms
45 0--------0 (206.214.251.137) 131.026 ms 146.013 ms 134.406 ms
46 0-------0 (206.214.251.142) 136.977 ms 134.188 ms 131.634 ms
47 0------0 (206.214.251.145) 133.170 ms 132.137 ms 131.775 ms
48 0-----0 (206.214.251.150) 131.519 ms 130.287 ms 132.893 ms
49 0----0 (206.214.251.153) 133.129 ms 138.519 ms 136.406 ms
50 0---0 (206.214.251.158) 138.966 ms 146.695 ms 142.437 ms
51 0--0 (206.214.251.161) 143.186 ms 132.973 ms 133.305 ms
52 0-0 (206.214.251.166) 137.930 ms 135.482 ms 136.323 ms
53 00 (206.214.251.169) 137.449 ms 135.504 ms 136.256 ms
54 i (206.214.251.174) 136.883 ms 135.939 ms 131.455 ms
55 by.ryan.werber (206.214.251.177) 131.137 ms 131.221 ms 131.316 ms
56 when.ccies.get.bored (206.214.251.182) 131.330 ms 131.044 ms 132.570 ms
57 read.more.at.beaglenetworks.net (206.214.251.185) 131.069 ms 135.185 ms 136.913 ms
58 read.more.at.beaglenetworks.net (206.214.251.190) 138.096 ms * 138.574 ms

 

Filed: General // No Comments

Submit for review

19 December 2012 21:13

If you run WordPress and have updated it, in my case to 3.5, you may have come across the bug where the “Publish” button disappears and all you can do is “Submit for Review”.

Now, a google search for this returns thousands of results, the fixes range from auto_increment values being full, harddrives being full, dodgy plugins, non-latest version plugins and DB corruption.

For me, non of those fixes worked. I started backing up the DB tables just by doing a simple duplication and on one table hit an error telling me that the default value (0000-00-00 00:00:00) wasn’t valid for a datetime column. I’m running MariaDB, but, it’s essentially MySQL – but fixed.

Turns out the fix is to remove NO_ZERO_DATE,NO_ZERO_IN_DATE from the sql_mode in the my.cnf. Restart and the issue goes away.

Interestingly on the previous WordPress version I had a plugin installed (Incorrect Datetime Bug Fix) but had to uninstall it because it broke 3.5.

WordPress need to get their date handling in order!

Filed: General // 1 Comment

Xcode Command Line Tools Install

22 October 2012 13:24

If you’ve upgraded to Mountain Lion you might not have Xcode’s command line tools installed anymore. Normally you can install via Xcode itself, but, if you’re going to be doing this remotely you’ll need command line instructions. It’s actually not to tricky. You’ll need a developer account for Apple though.

If you go to https://developer.apple.com/downloads/index.action, login and then (at the time of writing) download “Command Line Tools (OS X Mountain Lion) for Xcode – October 2” you’ll be on the way.

Next, mount the DMG:

$ hdiutil attach -mountpoint /Volumes/xc xcode451cltools_10_86938200a.dmg

Then run the installer:

$ sudo installer -pkg Command\ Line\ Tools\ \(Mountain\ Lion\).mpkg -target "/"

Lastly, detach:

$ hdiutil detach /Volumes/xc

Filed: General // No Comments

Migration

8 June 2012 15:58

You might notice that you’re now being bounced to archive.ianwinter.co.uk for all posts. I’m planning to move home over to ian.io. Right now you’ll end up in a loop of death until the holding page is finished.

Plan isn’t set, but, I might move over to jekyll, I might stay with wordpress, I might just not blog as much, I don’t know!

Either way it’ll change and I don’t have a new design yet.

Filed: General // No Comments

 
Twitter   •   About   •   Contact
©2016 Ian Winter. All Rights Reserved.   •   Powered by WordPress   •   Hosted at Memset