Devplace

Creating a Debian / Ubuntu package for a PHP PECL extension

Sat, 11 Aug 2012 16:06:05 +0200

Today I wanted to use the Inotify functions that are supplied by the PECL Package Inotify. It turns out that this PHP extension is not available in the (default) Debian or Ubuntu repositories. Instead of using the PECL command to install the extension, I wanted to create a native package that I could use to install/uninstall the package on multiple servers. This blogpost describes how I did this:

1. Downloading and unpacking the extension

Note that I will create a package for the Inotify extension in this blogpost, but this guide can be applied to any PHP (PECL) extension.

Create a working directory:
```

mkdir php-notify
cd php-notify
    
```

Download the extension:

wget 'http://pecl.php.net/get/inotify-0.1.6.tgz'

Unpack the extension:
```
tar -zxvf inotify-0.1.6.tgz
```
The current directory should now contain the extension archive (inotify-0.1.6.tgz) a package.xml file and a directory containing the source of the extension (i.e. inotify-0.1.6). You can verify this with:
```
ls -las
```

2. Preparing the extension for packaging

The source code should be in place. Now it's time to prepare the extension for packaging.

Make sure you have the needed tools installed to build the extension:


sudo aptitude update
sudo aptitude install build-essential checkinstall

Change directory to the extension source directory:
```
cd inotify-0.1.6
```
Run phpize to prepare the build environment for the extension:
```
phpize
```
Configure, compile and test the extension:
```

./configure
make
make test
```

These steps are also documented in the PHP manual at Compiling shared PECL extensions with phpize.

3. Creating a package

Now that we have compiled the extension, lets create a package for it! Creating a package is really easy now, all you have to do is issue the checkinstall command, the --install=no parameter prevents the package from being installed immediately. You should run this command as root, because otherwise files could end up having the wrong permissions.


sudo checkinstall --install=no

You can answer with "y" to the question about the ./doc-pak directory that does not exist. After that you can input a description for your extension. Then you're given an overview of the package to be build, you can change any of the properties. Change at least the name of the package, I changed it to "php5-inotify-custom". Hit ENTER to build the package now. The name of the created package is printed by the checkinstall script (php5-inotify-custom_0.1.6-1_amd64.deb in my case). Now you can install the package with dpkg:

sudo dpkg -i php5-inotify-custom_0.1.6-1_amd64.deb

After installation you can enable the module by creating a new file "/etc/php5/conf.d/inotify.ini" containing the text "extension=inotify.so" (make sure to restart your webserver if needed). If you don't want to create this file manually after installation of the package, read the next chapter.


sudo echo 'extension=inotify.so' > /etc/php5/conf.d/inotify.ini
sudo service apache2 reload

The package can be uninstalled with dpkg too:

sudo dpkg -P php5-inotify-custom

4. Creating a package that enables the extension

I wanted to create a package that would enable the the new module straight away. This is how I did this:

I created a config file for inotify.ini in the /etc/php5/conf.d directory. This directory is automatically scanned for (additional) configuration files for PHP in Debian and Ubuntu:
```

sudo echo 'extension=inotify.so' > /etc/php5/conf.d/inotify.ini
```
To include this file in the package, we must create a text file containing the path to this file:
```
echo '/etc/php5/conf.d/inotify.ini' > checkinstall_files.txt
```
Now run checkinstall like this and follow the steps that are outlined in the previous chapter:
```

sudo checkinstall --install=no --include=checkinstall_files.txt
sudo rm /etc/php5/conf.d/inotify.ini
```
You can skip the last step to enable the module.

If you run into errors or warnings when installing this package via dpkg, you should read Checkinstall configfiles bug at the bottom of this post.

Why did I build a package?

So why did I build a package instead of installing the extension with the PECL command? I did this because there are quite a few advantages in creating a (custom) package:

It can be installed and uninstalled via the system's own package manager (dpkg).
When you're running your own package repository it's really easy to install the package on multiple servers (via apt-get/aptitude).
When you're using a tool such as Puppet to manage your servers you can add this to the list of packages that must be installed.
The system is aware that the package is installed and which version of the package is installed. You can also define the package dependencies for your custom packages.

Checkinstall configfiles bug

After adding the config file (/etc/php5/conf.d/inotify.ini) to the list of files that should be included in the package I started getting warnings when installing (and removing) the package. After some searching, I found a blog entry called Bug in checkinstall regarding conffiles, that explains the warnings and provides a solution for this problem.

Updating Hyper-V drivers after a kernel upgrade on CentOS

Sat, 26 Dec 2009 10:54:52 +0100

A few days ago I was updating a CentOS machine that happens to be running on Microsoft's Hyper-V platform. Everything went very smooth. So I tried to reboot the machine, because I wanted to make sure that everything worked fine after a reboot.

The reboot

Apparently the kernel was updated too and the system did not boot with the new kernel. Since I have no access to the Hyper V environment itself, I needed to call my hosting provider (I only have shell access to the server). The restarted the machine and chose to boot the old kernel (which still worked). This was only a temporary solution because after a reboot the machine would still try to boot the new kernel.

Linux Integration Components

After some investigation I found out that I needed to install the Linux Integration Components for Hyper-V in the new kernel. It turns out that these integration components must be installed for each kernel that you want to boot into. There is one problem however, the integration components will only be installed in the currently running kernel by default. Since I have no access to the Hyper V environment and the machine has no ethernet connection without the components, I needed to find another way to install the components into the new kernel. I called my hosting provider again and they wanted to help me, but they would charge me around € 90,- for this (simple) task. I decided to find another solution myself.

Installing the Linux Integration Components into a non-running kernel

There are a few good tutorials for installing the Linux Integration Components in the currently running kernel, but I couldn't find proper documentation. It turns out that the scripts (and makefiles) that install the components make intensive use of the "uname -r" command to find out for which kernel the integration components should be installed. This means taht by default it's not possible to install the components into another kernel. I found the following solution:

Create a checkpoint for your Hyper-V machine (if you can), so that you can always restore the machine to the current state in case something goes wrong.
Find the kernel that you want to boot in by listing the installed kernels.
```
rpm -q kernel
```
In my case I got the following output:
```

kernel-2.6.18-128.el5
kernel-2.6.18-128.7.1.el5
kernel-2.6.18-164.9.1.el5
```
In my case the running kernel was "2.6.18-128.7.1.el5" and I wanted to boot into the newest kernel "2.6.18-164.9.1.el5".
Backup your linux integration components setup directory:
```
cp -R Linux_IC_V2 Linux_IC_V2-backup
```
And change directory to the Linux integration components setup directory (in my case the components are installed in /root/Linux_IC_V2):
```
cd Linux_IC_V2
```

The setup script assumes that you've already booted the new kernel when you are installing the new drivers. I think that it would be a lot easier if the kernel version could be passed as an argument to the script. But this is not the case so we need to adjust some files in the setup directory, you can find out which files need to be changed by using grep:

grep -iR "uname -r" *

In my case the following files where found:


drivers/dist/blkvsc/Makefile
drivers/dist/blkvsc/Makefile
drivers/dist/vmbus/Makefile
drivers/dist/vmbus/Makefile
drivers/dist/tools/scripts/updategrub.pl
drivers/dist/tools/scripts/updateinitrd.p
drivers/dist/Makefile
drivers/dist/Makefile
drivers/dist/Makefile
drivers/dist/storvsc/Makefile
drivers/dist/storvsc/Makefile
drivers/dist/netvsc/Makefile
drivers/dist/netvsc/Makefile
drivers/dist/makefile.common
setup.pl

Now we need to edit all those files (you can use your favorite editor) and make sure that we replace "uname -r" with the new kernel version. So in my case I replaced
```
shell uname -r
```
with:
```
shell echo "2.6.18-164.9.1.el5"
```
in the Makefiles. And in the perl scripts you can replace
```
`uname -r`
```
with:
```
"2.6.18-164.9.1.el5"
```
Now it is time to install the integration components in the kernel:
```
./setup.pl drivers
```
If the previous step was successful, we can check /boot/grub/menu.lst to make sure that we are booting the new kernel on reboot.
We should now be able to boot into the new kernel!
```
reboot
```

I hope this post was useful for you, because I have spent quite some hours tackling this specific problem. I have learned a lot about the Hyper-V platform in those hours though...

Contributing to the PHP manual - an introduction

Sun, 08 Nov 2009 16:26:06 +0100

There are many ways to contribute to the PHP programming language. You can contribute by writing code, reporting bugs, helping other users on the mailing lists, writing documentation etc. This article is a guide to start documenting for PHP. I think that one of the reasons that PHP is so widely used is because of the quality and completeness of the PHP manual. The documentation team can't keep up with the code that is being written for PHP, so a lot of functionality that is available in PHP (or its extensions) is still undocumented. This is especially true for functions and modules that were added to PHP in the last couple of years. I think that it is important to create proper documentation, because otherwise the new functionality will only be used by experienced PHP developers that are aware of the existence of the undocumented functionality.

How to start?

To start writing documentation you should think about what PHP functions, classes or extensions that you want to write documentation for. I think one can have several motives to start writing documentation for PHP. This list might help you decide what you want to document about, but is by no means complete.

You've used an extension in PHP that you've gotten pretty familiar with, but the documentation of this extension is incomplete.
You've encountered one or more bugs in the current documentation that you want to correct.
You want to contribute to PHP because you use PHP a lot, but you are not comfortable with programming in C. In this case the pages undocumented functions in the PHP manual or missing examples in the PHP manual are a good place to start.

After you've decided what you want to document, it's time to read the PHP documentation howto. This howto is not a definitive guide to start writing documentation for PHP, but it should be sufficient to get you started. The documentation team has switched to SVN a few months ago so there might be some references to CVS in the howto. After you've read the basic principles of the PHP documentation process, you can introduce yourself on the phpdoc mailinglist and start submitting patches to this list.

Karma

You need to earn some karma before you will receive direct write access to the phpdoc SVN Repository. To earn karma you can join the discussions on the phpdoc mailinglist and start submitting patches to the documentation on the list. After you've earned some karma you can apply for a SVN account.

Translating the manual

The PHP manual is available in many different languages. If you're not a developer, you can still contribute to the PHP documentation by translating the manual into your own language. You can start by posting a message to the phpdoc mailinglist in which you introduce yourself. It is also possible to subscribe yourself to a language specific mailing list. All documentation related mailing lists start with "php.doc.", you can view the available lists at news.php.net. Additional information can be found in the PHP wiki item "Working with translations".

What you should know about the PHP Manual

Note that this information can also be found in the PHP wiki and elsewhere on the net, but I thought it could be useful to summarize it here.

The PHP manual is written in docbook XML format with a few enhancements that are PHP manual specific.
The PHP documentation source resides in the "phpdoc" SVN module from the PHP SVN Repository.
PhD is the PHP tool that converts the docbook xml files to the various output formats of the PHP manual (currently PHP, HTML, PDF, CHM and man pages can be generated). Information about using PhD on windows can be found in an excellent blog post called "Setting up PhD on Windows" by Elizabeth Marie Smith.
The PHP manual is generated on a weekly base. The version that's used by the documentation team is updated four times a day and can be found at http://docs.php.net/.
If you happen to work for a hosting provider, you can help the PHP Community by mirroring the PHP Manual for your country.

Storing large values in MySQL fields with PHP

Tue, 29 Sep 2009 22:55:00 +0200

Today a colleague at work ran into a problem when trying to store file contents into a MySQL database table. We are using this solution for quite some time now and it has always worked pretty good for documents and files up to 15 MB. The main reason for us to put files in the database is that it is easier to migrate a website to another server because the database contains all the data that we need. The problem appeared at a customers' webserver that is running Fedora. First we thought there must be something wrong with our PHP code. But as the code worked fine on our servers (Debian) we soon found out that we couldn't store more than 1 MB in our content field in the database. After a little more investigation the only solution to the problem seems to be raising the mysql system variable max_allowed_packet to a value that is bigger than the largest content that should go into a single field. For Debian the default max_allowed_packet size is 16 MB, but since the max_allowed_packet is not set for Fedora, MySQL defaults to 1 MB. We have no access to the MySQL server configuration so we had to solve problem this on the client side (our PHP code). An example of how this can be done is given here.


$fileName = '/file/name/of/uploaded/file.bin';
$fileSize = filesize($fileName);

// $db holds the (ADODB) database connection
// Fetch current packet size
$packetSize = (int) $db->GetOne('SELECT @@max_allowed_packet');
if ($packetSize < $fileSize + 2048) {
	$sql = sprintf('SET @@max_allowed_packet=%d', $fileSize + 2048);
	$db->Execute($sql);
}

$db->Execute('INSERT INTO `file`(`filename`, `filesize`) VALUES(?, ?)', array($fileName, $fileSize));
$fileId = $db->Insert_ID();

// Here you can store the file contents in database
$handle = fopen($fileName, 'r');
while (($data = fread($handle, 524288)) && strlen($data) > 0) {
	// Update record with chunks of 512K, until the file is completely in the database
	// This prevents PHP from running out of memory
	$sql = sprintf(
		'UPDATE `file` 
		SET `contents` = CONCAT(`contents`, ?) 
		WHERE `id` = ?');
	$this->db->Execute($sql, array($db->Quote($data), $fileId));
}
fclose($handle);

// now set the  max_allowed_packet back to old value
$sql = sprintf('SET @@max_allowed_packet=%d', $packetSize);
$db->Execute($sql);

It is possible to store values up to 1 GB in a field in MySQL (as 1 GB is currently the maximum allowed packet size). It is however not advisable to use the above method using CONCAT to put large files in the database, CONCAT gets very slow when the size of the blob gets bigger. If you want to put large files in the database you probably want to take a look at the LOAD_FILE function. This is however more complex if MySQL doesn't run on the same server as your PHP code is running on. As you can see in the above PHP code the following MySQL queries can be used to retrieve and set the max_allowed_packet variable for the current connection.


-- Fetch maximum allowed packet size (in bytes)
SELECT @@max_allowed_packet;

-- Set maximum allowed packet size (in bytes)
SET @@max_allowed_packet=33554432;

We use the following table definition to store the files (I have left out some unimportant fields).


CREATE TABLE `file` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `filename` varchar(255) NOT NULL,
  `filesize` varchar(50) NOT NULL,
  `contents` longblob NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB CHARSET=utf8

Loading SyntaxHighlighter on demand

Tue, 01 Sep 2009 22:12:07 +0200

SyntaxHighlighter is a great JavaScript script, but it lacks one important feature in my opinion. For this blog, I needed a syntax highlighter for the code examples that will be shown here. I soon found SyntaxHighlighter as I was looking for an unobtrusive JavaScript solution. But there's one thing that I didn't like about it. You need to load all JavaScript files in advance even if you don't need syntax highlighting on a given page. This is not a problem when you're building static pages, because then you can just include the needed JavaScript files yourself. For this blog however I wanted SyntaxHighlighter to be loaded on demand without specifying for which page the SyntaxHighlighter should be loaded. So I came up with the nifty JavaScript solution below. It uses jQuery because jQuery is already loaded on all pages of this blog. But it shouldn't be so hard to do the same without jQuery.


$(document).ready(
	function() {

		// only load syntax hilighter when needed
		// So check for any HTML <pre> elements for which the class attribute starts with 'brush'
		if ($("pre[class^=brush]").length > 0) {
	
			var jsPath = themeUrl + '/js/hilighter/';
			var cssPath = themeUrl + '/css/hilighter/';
	
			// Defines which languages should be supported
			var brushes = ['Bash','Php','Plain','JScript', 'Sql'];
	
			var jsBrushes = '';
			for (var i = 0; i < brushes.length; i) {
				jsBrushes  = '<script src="' + jsPath + 'shBrush' + brushes[i] + '.js" type="text/javascript"></script>';
			}
	
			// Append SyntaxHighlighter scripts to the HTML head
			$("head").append(
				'<script src="' + jsPath + 'shCore.js" type="text/javascript"></script>' +
				  jsBrushes +
				  '<link rel="stylesheet" href="' + cssPath + 'shCore.css" type="text/css">' +
				  '<link rel="stylesheet" href="' + cssPath + 'shThemeDefault.css" type="text/css">'
			);
			// Apply SyntaxHilighlighter
			SyntaxHighlighter.all();
		}
	}
);

I think the script is pretty much self-explanatory. First we check if there are one or more PRE-elements having a classname that starts with "brush". If one or more of such PRE-elements exist in our HTML code, the appropriate SyntaxHighlighter CSS and JavaScript files will be loaded by adding script tags to the HTML head.