DECEMBER 2002 - Volume I - Issue 1
php|architect The Magazine For PHP Professionals
Zeev Suraski - Brad Young PERFORMANCE MANAGEMENT OPPORTUNITIES
This copy is registered to: Liwei Cui
[email protected]
Comparing PHP With the Industry at Large
Reviewed for you: Zend Performance Suite Davor's PHP Editor
Plus:
SNMP Management With PHP PHP nCurses Shell Programming Writing a Webmail System Distill PDFs For Free With PHP Create a DB Abstraction Layer
SUBSCRIBE AND
WIN!
In Colaboration with
The designers of PHP offer you the full spectrum of PHP solutions
Serve More. With Less. Zend Performance Suite Reliable Performance Management for PHP
Visit www.zend.com for evaluation version and ROI calculator
Technologies Ltd.
TABLE OF CONTENTS
php|architect Departments 4
| Editorial rants
Features 9
|
Performance Management Opportunities
INDEX
by Zeev Suraski and Brad Young
5
| New Stuff
14
|
Curses! Writing Cool Shell Applications With PHP
6
by Marco Tabini
| PHP-Win A Look at Davor’s
25
|
PHP Editor
SNMP Management With PHP by Harrie Hazewinkel
61
| Product Review Zend Performance Suite
34
|
Database Abstraction Layers
Version 2.5
65
| Tips & Tricks
A Concrete Look at by Maxim Maletsky
41
|
Creating A Visual Catalogue System with PHP and IE
by John W. Holmes
by Davor Pleskina
70
| For Your Reading
49
|
Writing a Web-based
Pleasure
PDF Converter
Book Reviews by the
by Marco Tabini
php|a team
72
| Exit(0); A War With No Enemies
December 2002 · PHP Architect · www.phparch.com
55
|
Directing Mail with qmail’s Maildirs by Jonathan Sharp
3
EDITORIAL RANTS
EDITORIAL
H
ere we are. After months of preparations—and a few sleepless nights—you hold in your hands the very first copy of php|architect! While most editors will spend their first editorial explaining at length the arduous path the brought the fruit of their labor to life, I feel I have a higher goal to aspire to. Getting php|a off the ground has introduced us to some genuinely new experiences—nothing beats being awake at two o’clock in the morning (when you couldn’t read the lettering off a truck if it hit you) trying to pick a highly legible font that will work both on the screen and in print. Still, I don’t think you want to hear about that. Let me, instead, give you an idea of how this issue is organized, so that you can take advantage of it to its fullest potential. We chose PDF and electronic distribution as a move that, we thought, would drive down our costs and, in turn, make it affordable for anybody in the world to subscribe or buy a single issue of php|a. Still, we recognize that there are some unique challenges related to using this distribution strategy, and we have taken a few steps that (we hope) will make it easier for everyone to enjoy our magazine. The PDF file that you have received has been designed both for on-screen viewing and for printing on a regular inkjet or laser printer. This way, you can look the contents over directly on your computer, and you can print whatever interests you—or the whole thing, if you like the feel of paper in your hands. In fact, although the PDF file itself
is encrypted and you should not redistribute it electronically, you can print as many copies of it as you like. If you decide that on-screen viewing is your bag, then here are a couple of pointers. Most of our articles are hot-linked to the Internet—whether it’s an e-mail address to contact an author or the URL of a website from which you can download some great information. Links are an opportunity for us to offer you something that a print magazine will never be able to—a direct connection to the world. Naturally, all our advertisements are also linked—so, show us your support and do visit our advertisers’ websites. They all offer great products. No, really, they do! To make the navigation easier, you will find a full table of contents, which can be accessed from within Acrobat’s bookmarks tab. Notwithstanding all these steps we’ve undertaken, PDF remains a “new” way of publishing a magazine, and we can only hope to get better with time. Your feedback in this process is crucial—we want to hear what we’ve done right, but, especially, we want to hear what we’ve done wrong, so that we can fix it. Drop us an e-mail at
[email protected] and let us know what you like and don’t like, what topics you want to see discussed in future issues and any comments you may have about this month’s articles. Happy reading!
December 2002 · PHP Architect · www.phparch.com
php|architect Volume I - Issue 1 December, 2002 Publisher Marco Tabini
Editors Arbi Arzoumani Marco Tabini
Graphics & Layout Arbi Arzoumani
Administration Emanuela Corso
Authors Arbi Arzoumani, Harrie Hazelwinkel, John W. Holmes, Maxim Maletsky, Davor Pleskina, Jonathan Sharp, Zeev Suraski, Marco Tabini, Brad Young
php|architect (ISSN 1705-1142) is published twelve times a year by Marco Tabini & Associates, Inc., P.O. Box. 3342, Markham, ON L3R 6G6, Canada. Although all possible care has been placed in assuring the accuracy of the contents of this magazine, including all associated source code, listings and figures, the publisher assumes no responsibilities with regards of use of the information contained herein or in all associated material.
Contact Information: General mailbox: Editorial: Subscriptions: Sales & advertising: Technical support:
[email protected] [email protected] [email protected] [email protected] [email protected]
Copyright © 2002 Marco Tabini & Associates, Inc. — All Rights Reserved
4
NEW STUFF
NEW STUFF
Open-source Gets a Rap-around As open-source software becomes more and more accepted as a viable alternative to its commercial counterpart in IT industry, it’s becoming painfully obvious that it’s necessary to build an infrastructure to facilitate the exchange of information between developers and customers. Thanks to its open nature, OSS lends itself well to customization— something that is normally unthinkable with commercial products. Since there is no licensing cost, companies find it easy to use opensource projects as the basis for larger initiatives that have very specific requirements. This results in ad-hoc solutions that are based on solid, well-tested code and can be deployed in a short time. Open-source developers are in the unique position of possessing the knowledge needed to successfully market customization and special features to their software
You’ve got a great software product and committed users, but you need additional funding to develop the features they require. packages. Unfortunately, they often suffer from a lack of exposure that would allow them to defray development costs of new features by offering such features to multiple customers at the same time. Marcheka Software Ltd. aims at filling the void between OSS vendors and clients through their Rap-X offering, which can be found at http://www.rap-x.com. Labeled as a “collaborative sales & support platform”, Rap-X works by allowing prospective clients to make requests for new features to be added to software packages in a collaborative way, so that multiple requests for similar changes can be merged into a single one and the development costs can be amortized among several players. With this approach, everyone ends up a winner: developers get the exposure they need and can open
December 2002 · PHP Architect · www.phparch.com
new markets for their products while still benefiting the public at large by keeping their products free. Customers, on the other hand, benefit from the fact that they can enjoy customized software at a fraction of the costs normally associated with it. Rap-x, which already counts five different opensource projects among its clients, is offered both as an ASP (as in Application Solutions Provider, not Active Server Pages!) product and through a more traditional licensing model. You can find its website here. PHP 4.3 Inches Nearer With the release of RC 2 at the end of November , the new PHP version 4.3 is one step from its final release. The new version of our beloved language includes several enhancements and performance optimizations compared to its predecessor. Our favorite one is the ability to fetch SSL-protected pages directly through the fopen() URL wrappers—a major step forward, in our opinion, toward making PHP ever more appealing for commercial applications. You can download PHP 4.3 RC2 from the PHP Group’s main page. Sessions, Anyone? Farpointer Technologies has released a new session management tool for Apache called iConductor. Yes, we know, PHP already supports session management. iConductor, however, is interesting because it does so independently of PHP (or any other language) by hooking directly into Apache. This makes it possible, for example, to manage sessions directly from within HTML pages, which, in turn, could mean lower overhead when processing pages. iConductor costs $1,295.00 and is available directly from Farpointer at http://www.i-conductor.com.
php|a
5
PHP-WIN
PHP-W WIN
Reviewed For You
Davor’s PHP Editor By Arbi Arzoumani
F
or a vim addict (recently converted to using gvim and the open-source CREAM extension) like myself, GUI IDEs are something to look at with an air of suspicious resentment. On one hand, they often look easy to use and user-friendly (something that can’t really be said about vim). On the other, however, my shell junkie friends and I fear that, as soon as you need to do something out of the ordinary, they will turn into rigid monsters that won’t allow you to move past their pre-set functionality. Luckily, I’ve had a chance to find out that it’s not necessarily always the case. After having been scarred for years by Microsoft’s Visual Studio IDE, I had a chance to use Zend Studio, which I found extremely powerful (particularly in its debug functionality, which are very valuable), although made a little slow by the fact that it’s based on Java. Still, even for its price, Zend Studio is extremely good—and works across multiple platforms. A few weeks ago, I came across Davor Pleskina’s PHP Editor (DPHPE from now on), which is available from his website at http://www.pleskina.com. Davor is a great guy (he’s also the author of this month’s article on catalogue management), and he suggested that I give his IDE a try. Firing Up DPHPE is extremely easy to install, although it doesn’t come with a setup program of its own. You do have
December 2002 · PHP Architect · www.phparch.com
to manually copy the files from the ZIP file you can find at Davor’s website into a folder on your PC and create the necessary shortcuts. I can’t see this as being a huge problem, since it’s safe to assume that you probably have some level of confidence with the OS if you’re interested in this product. Still, creating a setup program is not a particularly challenging endeavor these days—there are plenty of tools for that—and a more guided approach to installation would probably benefit most users. To give you an example, during my first attempt at running DPHPE, I simply executed the application’s main executable from directly within WinRAR (my compression application) without copying any supporting files. The results were not catastrophic—the application was running fine for the most part—but I had lost access to some functionality, such as keyword autocompletion.
DPHPE is also able to maintain a list of all the variables that are currently in scope in your script... Once installed, DPHPE takes up very little space on
6
PHP-WIN
Davor’s PHP Editor
your hard drive, which luckily also means that it uses very little RAM when it has to run. As much as this doesn’t seem to be a big problem these days, if your machine is running a number of different services at the same time, the last thing you need is another application that gobbles memory.
Another well-developed feature of DPHPE is the ability to create and maintain projects. Basic Features Where do I begin? DPHPE has a wide array of different features that are undoubtedly helpful for any developer. Most impressive of all, these features are implemented in a simple and efficient way, so that they actually help without getting in the way (at least most of the time). The editor that comes with the application features syntax highlighting (for HTML, PHP, Javascript and CSS), and its aspect is fully customizable—both
December 2002 · PHP Architect · www.phparch.com
from the point of view of the fonts used and of the colours in which the syntax is rendered. Also featured in the editor is the ability for auto-completion of keywords, which is particularly useful for a language as complex as PHP. It is also possible to have the application create a list of functions defined in your scripts, although this doesn’t always work well—for example, I had a comment that contained the word “function”, and the editor mistakenly recognized it as a declaration. DPHPE is also able to maintain a list of all the variables that are currently in scope in your script (although, unfortunately, it is unable to extend this capability to any include files you may be using) and to automatically pop up a window that allows you to choose a variable whenever you type the dollar sign. This feature tends to be a bit annoying after a while, but it can be turned off and the variable list can be used manually as needed. My only real beef with the editor is that I couldn’t find a way to word-wrap my long lines of code. Not everyone uses this feature, but I find it very convenient and not having it makes it a bit difficult to use the editor in a comfortable way.
7
PHP-WIN Project Management and Web Preview Another well-developed feature of DPHPE is the ability to create and maintain projects. A project is simply a repository of files but, contrary to a folder, it also possesses a number of properties that make it possible, for example, to preview entire web projects at the click of a mouse. What’s most interesting, however, is that the concept of projects lays the foundation for a number of future expansions, such as encoding, CVS or SourceSafe compatibility, global search and replace, and so on, that could all be accessible at the click of a mouse. Speaking of web preview, if you have Internet Explorer installed on your machine (which you must have, unless you’re running some ancient version of Windows), DPHPE is capable of interfacing to it and visualizing your PHP scripts by executing them through your web server. If you prefer, you can also launch IE separately rather than within the editor. Unfortunately, DPHPE does not feature an internal debugger—and I personally think
December 2002 · PHP Architect · www.phparch.com
Davor’s PHP Editor that this should be a very important future enhancement. Still, this application offers a great deal of functionality at a very convenient price—it’s free. What’s more, there is no big corporate presence behind it, so that it really evolves following the needs of its users, and Davor does a great job of listening. php|a
8
FEATURES
FEATURES
Performance Management Opportunities By Zeev Suraski and Brad Young Why Focus on Performance Management? A look at the various alternatives that exist in today’s complex IT world to improve server performance for sites that run PHP.
T
o start, let’s consider the reasons why we should be concerned with Performance Management. In a nutshell, it can be summarized with two simple goals: 1) Save Money 2) Happy Customers Although this may seem obvious, or even trite, it is important to keep in focus when evaluating and deploying performance-related products. To take it one level deeper, let’s understand the issues faced by IT managers and webmasters today:
· High expenses for hardware and software purchases to support http site traffic
· High expenses for database server hardware and software
· IT administration time and money spent to reconfigure and maintain server farms
· Low customer satisfaction resulting from unreasonable latency time and load time With these issues in mind, we can now more clearly attack the question of how to achieve improvements for our projects.
December 2002 · PHP Architect · www.phparch.com
As Yogi Berra famously stated, “It’s hard to make predictions, especially about the future.” For PHP, however, there are some things that are not so difficult to predict. Options For Performance Management There are many different ways through which you can squeeze more juice from your limited hardware server resources. These options include hardware, software, and networking alternatives. We’ll highlight each of these options below. Note that these alternatives are not mutually exclusive. Hardware Options
More Hardware - The most common indicator – aside from emails from angry site users – that raises a red flag about performance issues is maxed out CPU utilization. And when this happens, an obvious but costly solution is simply to buy more hardware. If you have two dual-CPU http server machines whose CPU’s
9
FEATURES
Performance Management Opportunities
are maxed out, then adding a third machine, with load balancing, will alleviate some of this load stress. It is worth noting that this solution works for steady state performance issues, but does not scale well for spiked performance issues. For example, if your site experiences dramatic increases in requests at specific times of the day, you’ll need to add enough hardware to cover the ‘worst-case’ load. (Actually, it is more accurate to call this the ‘best-case’. We want more traffic on our sites, after all.) Smarter Hardware – Similar to adding more hardware, adding smarter hardware can provide performance improvements. This includes the more obvious steps of having machines with faster CPUs, or more CPUs per machines. Beyond that, there is also hardware components such as disk drives with built-in memory caching that also help improve hardware performance. Hardware Devices – Another hardware option to improve performance is to utilize hardware devices that have built-in algorithm circuitry. For example, performing SSL encryption using software on your http server adds a significant stress to the CPU load. You can add a hardware card that performs this encryption without requiring any CPU time, thus lowering the effective load. Similarly, this can be done for gzip compression as well. Faster Code
It is clear that if your code is faster and more efficient, you’ll be able to handle more requests with the same hardware. Let’s explore how we can make code run faster. C Libraries– The massive adoption of PHP has come about mainly due to its ease of adoption – both from a cost perspective as well as a development perspective. But while PHP speed has improved tremendously, it is a fact that C code runs faster. So if you develop some of your more algorithmically-complex functions in C and call out to them from within your PHP scripts, you can speed up your code. Of course, by doing this, you give up some of the flexibility, transportability and ease of use that brought you to PHP in the first place. Write Better Code – Better coding practices, while tiresome at times to adopt, pay off in the long run. Always stay up to date and aware of the best ways to use PHP by following email lists from php.net and reading tutorial materials at sites like PHPbuilder.com and at the Zend Developer Zone (devzone.zend.com)—not to mention the pages of php|a, of course! Code Acceleration – PHP is an interpreted scripting language. This means that each time a PHP script needs to run, the core Zend Engine will parse that file and compile it in memory, then execute it. Since the PHP scripts themselves typically don’t change from December 2002 · PHP Architect · www.phparch.com
request to request, this parsing and compilation phase is repetitive and wasteful. Code Acceleration products available today can eliminate that redundancy by maintaining the compiled version of the file in memory. Output Content Caching
Imagine that your friend asked you what the headline of the New York Times was today. To respond, you go outside, walk two blocks to the nearest news-stand, read the headline of the New York Times, then come back and tell him what that headline says. (We leave it to you to understand why your friend would ask this, but just go with the analogy). Now, imagine that another friend calls you and asks the exact same question. Would you repeat the journey down to the news-stand? Unless you are madly in love with the salesperson, probably not. You are smart enough to identify that this new request is identical to the previous request, and hopefully you are also smart enough to remember what the headline was.
PHP is an interpreted scripting language. This means that each time a PHP script needs to run, the core Zend Engine will parse that file and compile it in memory, then execute it. Well, unfortunately your http server is not as smart as you, and it, figuratively speaking, is probably making way too many trips down to the news-stand (read: database server). This is where content caching comes in. The idea of content caching is straightforward: your server works hard to generate some results to a specific request. Then it keeps those results on hand, and if it gets a similar request, it re-uses the previously generated results. It is interesting to note that caching takes place at the http server level and reduces load on that server, but an even bigger load savings is experienced on the database server, which sees dramatically less SQL requests coming in. The concept is simple enough, but with all things programmatic, the devil is in the details. The challenge mainly boils down to: · Cache Hit Identification – How does the cache determine if this is indeed a similar request, or is it a different request (i.e. Is my second friend asking for the same headline, or is he asking
10
FEATURES
Performance Management Opportunities Figure 1- Content Caching Schematic
about the New York Post, or is he asking about yesterday’s New York Times, or is the headline different for him because he gets the national edition instead of the local edition)? · Cache Management – How does the cache manage the information results that are onhand, purge old information, and prevent over-use of system resources (i.e. I can’t remember every headline of every newspaper from every day without my head exploding, so which is the best to forget)?
Now let’s look at a few ways of achieving content caching: File-level Content Caching – If you have web pages that appear the same to many users, you can store the entire results of the URL request coming in to your http server, as shown in Figure 1. For example, imagine a page that lists news headlines, such as cnn.com. New headlines are added, perhaps at a rate of one new headline per 15 minutes. CNN could cache the headlines page or at least 5 minutes. Now, if 1,000 page view requests come in during
Figure 2 - Partial Page Caching
December 2002 · PHP Architect · www.phparch.com
11
FEATURES
Performance Management Opportunities
that 5 minute period, only one actual PHP execution and one database access will occur, a dramatic decrease compared to the 1,000 that would otherwise occur. All the other requests are served directly from the cache, as if it was a static html file. And by having a time limit to this cached file, the worse case scenario Figure 3 - Database Intermediary
“Welcome, UserName” problem. In our headline listing page, the main portion of the page is the same for each user, but now we add a personalization message at the top of the page. We can no longer cache the entire file, unless we don’t mind that users named Judy, John, and Jade will get a page that says “Welcome, Joe”, the first person to request the URL.
PHP 5.0, with its new Zend Engine 2, will include new object handler capabilities that will not only make coding easier, it will also speed up application performance.
for content freshness would be that it would take a new headline 4 minutes and 59 seconds to make it to the site. When exploring solutions, you’ll want to make sure that there is sufficient flexibility for determining what should be considered a ‘cache hit’. Besides the basic URL address, you might also take specific cookie variables, GET parameters, or global http variables into consideration. For example, you might want to cache a different version of your home page for each geographic region, or type of user. Partial PageCaching – Solving the “Welcome, UserName” Problem– The benefits of file-level content caching are clear. But often, some portions of the page are personalized on a per-user basis. Let’s call this the
The solution here is partial page caching. We still can cache the majority of the page, which in this case also uses the most resource-intensive database queries. This is done at an API level, in which the caching of data segments is maintained and reused programmatically, as shown in figure 2. Database Intermediary Caching– A third option for caching of results involves separating your PHP code from the database, as shown in Figure 3. Instead, have a separate timed job that pulls content from the database and creates chunks of html files that the PHP scripts then can read in the output results that they are creating. This solution adds some complexity with regards to system configuration and code dispersal, but the results will sometimes make it worthwhile.
Connect with your database Publish your data fast with PHPLens PHPLens is the fastest rapid application tool you can find for publishing your databases and creating sophisticated web applications. Here’s what a satisfied customer, Ajit Dixit of Shreya Life Sciences Private Ltd has to say: I have written more than 650 programs and have almost covered 70% of MIS, Collaboration, Project Management, Workflow based system just in two months. This was only possible due to PHPLens. You can develop high quality programs at the speed of thinking with PHPLens
Visit phplens.com for more details. Free download. December 2002 · PHP Architect · www.phparch.com
12
FEATURES
Performance Management Opportunities
Database Optimization
Data Models and SQL Tuning – As programmers, we often find ourselves focusing on our area of expertise – coding. However, even the fastest optimized algorithm as written in PHP can’t make up for the time wasted by poorly written SQL statements or queries to databases that are not modeled or indexed properly. Don’t fall into this trap. As mentioned earlier under the subject heading of “Write Better Code”, take the time to learn about how to get the most out of your database server. If you have a DBA in house, use his or her expertise. As with content caching, a little effort can go a long way when you tweak your database usage.
Also, identify the resources and infrastructure you have available. Once you measure these parameters, selecting the various products or methodologies mentioned above becomes much easier. Enterprise Performance – Future Directions As Yogi Berra famously stated, “It’s hard to make predictions, especially about the future.” For PHP, however, there are some things that are not so difficult to predict.
· First, Apache 2.0 adoption may start to grow,
Data Compression
Gzip Compression – “Performance problems” is a big umbrella. Sometimes, the performance issue is not necessarily server throughput or CPU utilization, but rather end-user experience of download time. Slow download may result from either large data transfer, or overutilization of server bandwidth. In both cases, you will improve end-user performance experience by compressing your data. Most browsers today can receive compressed contents, and will decompress automatically. Using output compression is as simple as setting zlib.output_compression = on in php.ini. Browser detection will determine if gzip support is built in, or if the original plain text should be sent. However, it is important to note that this adds some additional CPU burden to your server, which must perform a gzip compression for each result being delivered. This can be overcome by using a compression solution that is integrated with content caching. In this case, the compressed version is cached, and not regenerated each time. Network Infrastructure
While it isn’t directly related to PHP dynamic content, it is worth noting that network optimization solutions, such as those provided by Akamai , can boost content delivery time. This can be especially useful if image download and delivery time is your performance bottleneck. Which Should You Choose? As stated in the opening of this article, these options are not necessarily mutually exclusive. So, how do you make choices on how to boost your performance? Well, in the words of the anthropologist Margaret Mead, “Always remember that you are absolutely unique, just like everyone else.” First, identify what specific problems you are having.
December 2002 · PHP Architect · www.phparch.com
·
·
when the Apache stability as well as the PHP stability on top of Apache 2.0eventually reaches the necessary level. We may see some boosts to performance from Apache 2.0’s multithreaded capabilities, on platforms such as Solaris. PHP 5.0, with its new Zend Engine 2, will include new object handler capabilities that will not only make coding easier, it will also speed up application performance. In the database world, newer extensions and support for MySQL 4 and Oracle will likely give another shot in the arm for database throughput. Why Performance Management Matters
We opened this article explaining the benefits that Performance Management brings you and your organization: Save Money and Happy Customers. But the importance runs even deeper. Collectively, the PHP community is proving every day that PHP is ready for front-line, enterprise-level applications. Implementing performance management solutions such as those highlighted here bring us further towards the recognition from the marketplace that PHP is here to stay. php|a
Zeev Suraski has been working for over five years on the PHP project. Along with Andi Gutmans, he started the PHP 3 and 4 projects and wrote most of their infrastructure and core components, thereby helping to forge PHP as we know it today and attracting many more developers to join the movement. Zeev is a co-founder and CTO of Zend Technologies Ltd, the leading provider of development and performance management tools for PHP-enabled enterprises. Brad Young leads the Product Management team at Zend, and has over 15 years experience in on-line database and XML systems deployment. You can reach both Zeev and Brad via Zend's website at www.zend.com.
13
FEATURES
FEATURES
Curses! Writing Cool Shell Applications With PHP By Marco Tabini While PHP is most widely used as a Web development language, I find that it works very well as a shell tool, too. In fact, I’d have to say that PHP is my favourite shell language, and that I prefer it over typical shells or even Perl (although the latter is rather unbeatable when you have to process large data files).
I
n a world of web-based management systems and graphical user interfaces, it’s comforting to find out that shell programming is still regarded as an important piece of the IT puzzle. Don’t get me wrong—I’m not saying this merely out of nostalgia for the times when I had to make do with 48kB of RAM for operating system, program and data, or had to wrestle a few cycles of program execution out of a mainframe. Text-based applications are still very useful in a number of situations, particularly when simplicity is called for or a GUI would be oppressively slow to deal with (such as when you’re trying to manage a web server that is heavily loaded and cannot spare much bandwidth for you). Shells, particularly (but not exclusively) in the UNIX world, are very much alive and kicking, and have advanced significantly in complexity and functionality over the past few years. nCurses - A Library For Cool Shell Apps Think “shell application” and the image of a dull, black-and-white text screen comes to mind. Luckily, most terminals are actually capable of a lot more than just displaying text: they provide a variety of colours and effects (such as underlined or bold—”bright”— text), mouse compatibility and much more. It’s normally rather difficult, however, to take advantage of these capabilities for a number of reasons. First, not all terminals are created equal—there are several different stan-
December 2002 · PHP Architect · www.phparch.com
dards available, each with its own commands and capabilities. Second, because terminal systems have to work over simple serial communication lines, the commands they support are often intricate combinations of ASCII characters. Since shell applications are usually written as a quick and easy solutions to simple problems, the developers rarely have the time and patience to bother with adding the “frills” to their interfaces. Those of you who, like me, have spent their youth wasting their allowances on telephone bills as a result of the expensive hobby known as “BBS”, however, know what terminals are capable of. Remember the cool menus, the dazzling animations and the colourful interface? Compare that with your average shell application and, well, I’d take the BBS over it every day. On UNIX systems, the capabilities of terminals are stored in the termcap database, a file usually found in the /etc directory that is used by applications such as the original vi editor. Termcap contains a description of each terminal that is recognized by the system (for example, “dumb” or “ansi-compatible”), together with the commands it supports. Termcap, however, is not in a very “developer-friendly” format. In fact, I think it’s fair to say that using it in REQUIREMENTS PHP Version: Latest PHP CVS Build O/S: nCurses-compatible O/S Web server: N/A Support Tools: nCurses Library
14
FEATURES
Curses! Writing Cool Shell Applications With PHP
an application with no outside help would require a major effort, since you would only have to interpret its contents, but also write the necessary functions to handle tasks like cursor positioning, text attributes, window management, and so on. It was probably out of desperation that, around 1980, a student at UC Berkley named Ken Arnold developed a series of routines that provided an essential framework for terminal interface development and collected them in what became known as the “curses” library. Curses then became the basis for Rogue, possibly one of the first-ever Role Playing Games, initially developed for BSD Unix but eventually ported to pretty much any operating system ever conceived. If you have never played Rogue, I suggest you give it a try— not only it’s a thoroughly enjoyable and never repetitive game, but it also gives you an idea of how engaging a simple text-based user interface can be. You can find Rogue in RPM format at : http://www.rpmfind.net/linux/rpm2html/ search.php?query=rogue I should warn you that you will find it highly addictive; your productivity will probably drop to zero for a while, but you’ll have lots of fun in the bargain.
• When you need to deploy a web-serving framework capable of delivering millions of views and transactions per day... • When you need to construct complex document and task workflows that match your organizations' needs... • When you need to create a world wide streaming network capable of 99.99% up time... • When you need to protect your country's sovereignty by avoiding "random" changes in software licensing that can affect the price of each transaction your citizens make... • When you need the level of security that only expertise and transparency can provide you...
The Curses library was picked up by Bell Labs, which included it in its System III version of UNIX, introducing several changes to termcap in the process. Termcap’s new format, called terminfo, offered more functionality. Additional development was done through the 1980s, and in 1993 the package was finally renamed ncurses. Ncurses and PHP The ncurses library provides a number of features, such as: · Formatted input and output on a variety of terminals · Support for colours and various text styles, such as underline, bold, and so on · Support for windowing · Support for mouse input PHP support for the ncurses library is still very much in its infancy, and is based on the GNU implementation of the library, available via FTP at: ftp://ftp.gnu.org/pub/gnu/ncurses
• When you need Single-Sign-On across multiple vendors serving a diverse constituency... • When you need to reduce your reliance on proprietary software solutions and realize the promise of your existing investments... The Tribal Knowledge Group's demonstrated credentials in the Open Source Software community enable them to provide you with decades of software engineering, architecture, security and strategy expertise via an unprecedented think-tank of Internet technology and open-source experts. Contact us at
[email protected] to learn how the Tribal Knowledge Group can help maximize your existing investment for the future.
Proven Open Source Expertise
December 2002 · PHP Architect · www.phparch.com
15
FEATURES It was originally introduced by Hartmut Holzgraefe and Georg Richter with version 4.1.0 of PHP and is still considered highly experimental, to the point that the official PHP documentation covers the ncurses functions only very quickly and in a still incomplete fashion. Luckily, the ncurses extension functions follow their original counterparts very closely, which actually makes the process of creating an ncurses-powered PHP script very easy (this is something that the authors of the extension must have undoubtedly known, so that they have—correctly, in my opinion—put all their efforts in building out the extension). Ncurses is actually supported on a number of platforms, including all the UNIX derivatives, cygwin and BeOS. Obtaining the Right Version of PHP The problem with making ncurses run on your server is that, in order to gain access to enough functionality, you will have to download the most recent version of PHP from the public CVS tree. There is no way around this problem, as the stable releases available to this point do not yet include the latest code, which is unfortunately necessary to perform most of the basic ncurses tasks. Luckily, obtaining the latest version of PHP is not difficult at all. You can get it directly by issuing a couple of CVS commands, which are clearly explained in the manual:
cvs -d :pserver:
[email protected]:/repository login cvs -d :pserver:
[email protected]:/repository co php4
If all goes well, you should end up with a sub-folder of your current directory called php4. Simply cd into it and configure PHP as you would normally, ensuring that you include --with-ncurses to your invocation of configure, and that you compile the CLI version of PHP—after all, I’m afraid that you would have little use for the ncurses extension in a web application. Our First ncurses Application It’s now time to test whether the installation went smoothly by executing the simple PHP script that you can see in Listing 1. This script is designed to be run from the shell. As such, it includes a “shebang”, that is, a specially-formatted comment (always the first line of the script), that instructs the shell as to which interpreter this file should be executed through. This approach works only on UNIX-type shells, however, so if you’re running Windows you’ll have to use the cygwin shell rather than the DOS command prompt. December 2002 · PHP Architect · www.phparch.com
Curses! Writing Cool Shell Applications With PHP If you do not receive any errors, you should see the words “Hello World!” written in red text over a cyan background appear at the top of your screen for a few seconds and then disappear. If something goes wrong, there’s a good chance that your shell will become unusable—don’t panic, this is quite normal, as we’ll see in a few moments. As you can see, even a simple Hello World application requires quite a bit of code if you write it using ncurses. You not only have to initialize the library using the ncurses_init() function, but you have to create a combination, or “pair”, of each foreground and background colours that you want to use in your application. This, in turn, will only work if the terminal you are using supports colours to start with, and the ncurses_has_colors() function provides the script with that information. Even writing a string to the screen requires a call to a specialized ncurses function, ncurses_addstr(). The latter is actually part of a family of functions that make it possible to manipulate text on the screen. Incidentally, you will also have noticed that it’s necessary to forcefully refresh the screen in order for any updates made to it to become visible. This particular aspect of ncurses programming is actually designed so that only the minimum amount of commands are sent to the screen, and only when necessary—thus reducing the amount of data sent to the terminal by as much as possible. With today’s fast connections, this may sound a bit ridiculous, but you have to understand that ncurses must be able to work efficiently even over slow serial connections—and a bit of optimization never hurts. Thus, for the most part, an application writes to a “virtual screen” and then issues a call to ncurses_refresh() when all of its updates are complete and it is ready to show its output to the user. Finally, the ncurses library must be explicitly terminated by calling ncurses_end(). This is necessary because ncurses captures the terminal and creates its virtual screen over it. As a result, exiting from a script without also terminating ncurses results in a terminal that, in most cases, is completely unusable. The problem of course, is that sometimes scripts end unexpectedly, as it is often the case when errors occur. For this reason, it’s always a good idea to create your own error handler that explicitly calls ncurses_end() before aborting your script. This will not protect you from all kinds of problems (parse errors will still cause PHP to exit without calling your handler, for example), but it will take care of at least some cases. With the exception of windowing functions, which we’ll examine in the next section, all ncurses functions return true if an error has occurred (they actually return an integer value different from NCURSES_ERR, which is defined as zero).
16
FEATURES
Curses! Writing Cool Shell Applications With PHP Windowing
One of the most interesting characteristics of ncurses is the ability to create and manage multiple windows. Although they cannot overlap each other, each window possesses its own “virtual screen” and can be modified and refreshed independently of its other counterpart on the terminal. As an example, let’s take a look at Listing 2, which introduces a number of new ncurses functions. The first one is ncurses_getmaxyx(), which returns the dimensions of the virtual screen. You have probably noticed that, in this function, the height is listed in the function call before the width—something that may appear unnatural to someone who is used to working with GUI systems. All ncurses functions follow this convention, however, and it does take a while to get used to it. Also, the call to ncurses_getmaxyx() at the beginning of Listing 2 uses a constant called STDSCR. This constant describes, by convention, a pointer to the main virtual screen, which is, for all purposes, a window of its own right. In fact, most of the ncurses functions that manipulate the virtual screen also have a windowspecific counterpart—for example, ncurses_addstr() and ncurses_waddstr(). In most cases, the ncurses library implements the virtualscreen version of these functions as simple macros to the window versions with the hardcoded STDSCR parameter. The ncurses_newwin() function is used to create a new window. Its syntax is as follows: resource ncurses_newwin ( int $height, int $width, int $y, int $x );
Its result is an ncurses resource that you can use in subsequent calls to other ncurses functions that can manipulate the contents of your newly created window, such as ncurses_wborder, which—you guessed it—creates a border around the frame of our window: int ncurses_wborder ( resource $win_resource, int $left_side, int $right_side, int $top_side, int $bottom_side, int $top_left_corner, int $top_right_corner, int $bottom_left_corner, int $bottom_right_corner );
December 2002 · PHP Architect · www.phparch.com
With the exception of the first parameter, which identifies the window where the border is to be drawn, each of the other parameters represent the characters that the function must use to build the border. If you leave all these parameters as NULL or zero, ncurses_wborder() will create a standard (and classy) border around the window. If, for example, you call the function as follows: ncurses_wborder ( $win1, ‘|’, ‘|’, ‘-’, ‘-’, ‘+’, ‘+’, ‘+’, ‘+’ );
You will obtain a slightly less pleasant—but just as functional—border. Writing into a window is performed through a number of functions. The simplest one is ncurses_waddstr(), which is functionally equal to ncurses_addstr(), with the exception that it takes a window resource as the first parameter. In the case of Listing 2, however, I have used a slightly more complex version of it called ncurses_mvwaddstr(), which at the same time moves the cursor to a pre-defined position and prints out the string passed to it: int ncurses_mvwaddstr ( resource $win_resource, int $y, int $x, string $str );
As you may have noticed, the x and y coordinates passed to this functions reference the top-left hand corner of the window—not of the screen. There is, naturally, an equivalent of ncurses_mvwaddstr() that works with the virtual screen and is called ncurses_mvaddstr(). This, in turn, is functionally equivalent to calling ncurses_mvwaddstr() as follows: ncurses_mvwaddstr ( STDSCR, $y, $x, $str );
Finally, we come to the visualization of windows, which is achieved by calling the ncurses_wrefresh() function. Keep in mind that ncurses does not handle windows the same way a GUI system like
17
FEATURES Windows or XWindows would; this means that the overlapping of windows and features like z-order are not normally provided to the developer (although another portion of ncurses called panels, which is also being implemented in the PHP extension, does provide exactly this functionality). Thus, wrefresh() should simply draw a particular window on the screen, as if it were in focus and with complete disregard for whatever other windows may overlap with it. I say “should” because in reality different versions of ncurses behave in different ways—as such, it’s best to either use only tiled windows, or to use one window at a time on top of the virtual screen. Thus, calling wrefresh() will cause a window to be drawn ontop of the virtual screen, in our cases advising us that its contents are rather messy. Another interesting characteristic of ncurses is that it provides no scrolling capabilities. When you reach the end of the screen (or of a window), text is simply discarded. Depending on the type of application that you are writing, this could be a major problem—but it can be easily solved as we’ll see later on. A Bit of Colour and a Sprinkle of Fairy Dust As marketing people have known for years, colour makes all the difference. In a well-designed interface, proper colour management can make any application easier to use (except for vim, which was obviously written by aliens) and more pleasant to the eye. The ncurses library provides an extensive set of functions that deal with the aspect of text—not only with its colour, but also with its intensity and style. For example, Listing 3 produces text in quite a variety of styles. Because colours can only be rendered on some terminals, the ncurses extension provides the ncurses_has_colour() function to determine whether colours can, in fact, be used on the current terminal. Also, colour rendering has to be enabled by calling the ncurses_start_color() function. The ncurses library, as I mentioned above, handles colours by means of “pairs”, each representing the combination of a foreground and background colour. Each pair is defined using the ncurses_init_pair() function: int ncurses_init_pair ( int $pair_id, int $foreground_colour, int $background_colour );
The ncurses extension provides a number of pre-set colour definitions, which are shown in Figure 1. On some terminals, it is also possible to create custom December 2002 · PHP Architect · www.phparch.com
Curses! Writing Cool Shell Applications With PHP colours by changing the RGB values associated with a particular pre-defined colour (you could finally have your own opportunity to invent the “greenest blue” ever made!) through a call to ncurses_init_color(), whose syntax is as follows: int ncurses_init_color ( int int int int );
$color, $rgb, $green, $blue
A call to ncurses_can_change_color(), which takes no parameters, will tell you whether you can, indeed, change the appearance of a particular colour. Figure 1 NCURSES_COLOR_BLACK NCURSES_COLOR_WHITE NCURSES_COLOR_CYAN NCURSES_COLOR_RED NCURSES_COLOR_YELLOW NCURSES_COLOR_BLUE NCURSES_COLOR_MAGENTA NCURSES_COLOR_GREEN
Black White Cyan Red Yellow Blue Magenta Green
Attributes are handled in a very similar fashion to HTML. They can be turned on using the ncurses_attron() function (or ncurses_wattron() for a window), or off through a call to ncurses_attroff(). A list of available attributes is shown in Figure 2. Figure 2 A_NORMAL A_STANDOUT A_UNDERLINE A_REVERSE A_BLINK A_DIM A_BOLD A_PROTECT A_INVIS
Normal font Standout (reverse) Underlined Reverse screen Blinking Text Dimmed Bold Protected Invisible
Similarly to colours, there is a way to determine whether a particular attribute is supported by the current terminal. The ncurses_termattrs() function takes no parameter and returns a logical OR of all the attribute values that the terminal can handle.
18
FEATURES
Curses! Writing Cool Shell Applications With PHP Listing 1
1 #!/usr/local/bin/php 2 3
Input Management As I’m sure you can imagine, a good user interface without input capabilities is a bit like a war with no enemy—you could make one but it would be rather dull and quite pointless. The ncurses library offers a number of facilities for both keyboard and mouse input, although we will only focus on the latter as far as this article is concerned. The most commonly used form of keyboard input is ncurses_getch(), which retrieves a single character from the input stream. Under normal circumstances, that character is also outputted to the screen, but you can prevent that from happening by calling the ncurses_noecho() function. For example, take a look at Listing 4, which shows a simple script that can be used to control the Apache web server through a series of simple commands. As you can see, there are a couple of interesting functions. First, I’ve created a simple msgbox() routine
December 2002 · PHP Architect · www.phparch.com
that takes an arbitrary amount of text and displays it in an ordered fashion on a window of its own. This corresponds to a simple dialogue window in a GUI environment. You will notice that I have used the wordwrap() function to ensure that the text is always properly centered and never interferes with the window’s borders. The lack of a couple of ncurses functions (in particular wsyncup() and wsyncdown()) in the PHP extension make it slightly awkward to manage windowing properly, so that the application is forced to manually refresh the screen every time a dialogue box is displayed. The ncurses_getch() function is called in an infinite loop until the user presses the “q” key and exits. The result value of ncurses_getch() is not, however, a character, but an integer value. This lets you manage “special” characters, such as Page-up and Pagedown, more easily and conveniently—but you will have to convert the integer value to a string if you want to interpret it as a character. More complex input can be
19
FEATURES
Curses! Writing Cool Shell Applications With PHP
performed by calling the ncurses_instr() function, which reads an entire string from the input and returns it. It’s interesting, at least in my opinion, to note that even a simple script like this one can be set as the default command for a particular user account, thus depriving it of shell access but still giving it the opportunity to perform certain operations—in this case, stopping and starting Apache. A More Complex Example Let’s now move on to a more audacious attempt at using the ncurses library to produce a complete shell-
based application. Listing 5 contains a self-contained text file viewer that allows for up and down scrolling (both line by line and page by page), can split and display lines of text that span multiple lines (without using wordwrap, this time) and displays a status line at the bottom of the screen. This script introduces the ncurses_move() function, which is used to move the cursor to a particular location (in fact, ncurses_mvwaddstr() is essentially the equivalent of a call to ncurses_move() followed by a call to ncurses_waddstr()). You’ll notice that the most complicated action performed by this script is the tracking of the position of the screen’s output within the text file. For the script to
Listing 2 1 #!/usr/local/bin/php 2 3
Continued On Page 21...
December 2002 · PHP Architect · www.phparch.com
20
FEATURES
Curses! Writing Cool Shell Applications With PHP
stop scrolling at the end of the file, it is necessary to track both the position within the file that corresponds to the top of the screen (which we need so that we can refresh it), as well as the position that corresponds to the bottom of the screen. This is made more difficult by the fact that there is no direct correspondence between the width of the screen and the length of each line of text, so that a single line of text can in fact correspond to multiple lines on the screen—but we still want the script to scroll by one screen line at a time, so that the script has to track the correspondence between screen display and file. To make things a bit easier, I have separated the file in its individual lines by using the file() function. Clearly, it would have been much simpler to load the file in memory and then use wordwrap() on it to
Listing 2: Continued From Page 20... 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
// Refresh the window, thus making it visible ncurses_wrefresh ($win1); // Wait a while sleep (2); // Destroy the window ncurses_delwin ($win1); // Exit ncurses_end(); ?>
Listing 3 1 #!/usr/local/bin/php 2 3
December 2002 · PHP Architect · www.phparch.com
Continued On Page 24...
21
FEATURES
Curses! Writing Cool Shell Applications With PHP
ensure that there would be a one-to-one correspondence between the screen and the file. However, you can take this script and turn it into a full-fledged editor with relatively minor modifications—whereas if it were based on the wordwrap() trick it would be much more difficult to do so.
CVS tree of PHP, while there is no support for the form library. However, they are both very important, as they can greatly simplify the development of complex shellbased applications. For an example of their full capabilities, take a look at the Lynx text-based web browser, which uses both libraries to their fullest potential.
Where To Go From Here There is much more to the ncurses library than what we’ve been able to explore here. In particular, recent versions of ncurses include two libraries, panels and form, that can be used to manipulate overlapping (and scrollable) windows and form fields respectively. The panels library is already partially implemented in the
php|a
Marco Tabini is co-editor of php|architect. He spends most of his time lurking around on the PHP mailing lists and on the php|a website doing is best to confuse fellow programmers who are in trouble. You can reach him at
[email protected].
Listing 4 1 #!/usr/local/bin/php -q 2 3 (strlen ($text) + 4)) 40 $win_w = strlen ($text) + 4; 41 else 42 $text = wordwrap ($text, $win_w - 4); 43 44 $string_data = explode ("\n", $text); 45 46 if ($win_h > (count ($string_data) + 4)) 47 $win_h = count ($string_data) + 4; 48 49 // Create a new window 50
Continued On Page 23... December 2002 · PHP Architect · www.phparch.com
22
FEATURES
Curses! Writing Cool Shell Applications With PHP Listing 4: Continued From Page 22...
51 $win = ncurses_newwin ($win_h, $win_w, ($height - $win_h) / 2, ($width - $win_w) / 2); 52 53 // Apply a simple border 54 55 ncurses_wborder ($win, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); 56 57 // Type all the information 58 59 for ($i = 0; $i < min ($win_h - 4, count ($string_data)); $i++) 60 ncurses_mvwaddstr ($win, $i + 2, ($win_w - strlen ($string_data[$i])) / 2, $string_data[$i]); 61 62 // Refresh window 63 64 ncurses_wrefresh ($win); 65 66 // Wait for input 67 68 ncurses_getch(); 69 70 // Destroy window 71 72 ncurses_delwin ($win); 73 } 74 75 // Creates the main menu 76 77 function main_menu() 78 { 79 // Clear screen 80 81 ncurses_erase(); 82 83 // Create main menu 84 85 ncurses_color_set (2); 86 ncurses_attron (NCURSES_A_BOLD); 87 center_window (STDSCR, 2, "Main Menu"); 88 center_window (STDSCR, 3, "---------"); 89 ncurses_attroff (NCURSES_A_BOLD); 90 91 center_window (STDSCR, 4, '(1) Stop Apache'); 92 center_window (STDSCR, 5, '(2) Start Apache'); 93 center_window (STDSCR, 6, '(3) Restart Apache'); 94 center_window (STDSCR, 8, '(Q) Quit'); 95 96 ncurses_refresh(); 97 } 98 99 // Initialize ncurses 100 101 if (ncurses_init()) 102 die ("Unable to initialize ncurses!"); 103 104 // Initialize colour, if available 105 106 if (ncurses_has_colors()) 107 { 108 // Initialize a simple colour pair 109 // And select it 110 111 ncurses_start_color(); 112 113 ncurses_init_pair (1, NCURSES_COLOR_CYAN, NCURSES_COLOR_BLACK); 114 ncurses_init_pair (2, NCURSES_COLOR_RED, NCURSES_COLOR_BLACK); 115 } 116 117 // Turn off echo 118 119 ncurses_noecho(); 120 121 // Create main menu 122
Continued On Page 24... December 2002 · PHP Architect · www.phparch.com
23
FEATURES
Curses! Writing Cool Shell Applications With PHP LISTING 3: Continued From Page 21...
50 51 52 53 54 55 56 57 58 59 60 61 62
// Refresh the screen ncurses_refresh(); // Wait a while sleep (4); // Exit ncurses_end(); ?>
Listing 4: Continued From Page 23... 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155
main_menu(); while (true) { $c = ncurses_getch(); $s = chr ($c); switch ($s) { case '1' : msgbox (shell_exec ("apachectl start") . "\n\nPress a key to continue"); main_menu(); break; case '2' : msgbox (shell_exec ("apachectl stop") . "\n\nPress a key to continue"); main_menu(); break; case '3' : msgbox (shell_exec ("apachectl restart") . "\n\nPress a key to continue"); main_menu(); break; case 'Q' : case 'q' : ncurses_end(); exit; break; } } ?>
December 2002 · PHP Architect · www.phparch.com
24
FEATURES
FEATURES
SNMP Management With PHP By Harrie Hazewinkel In the last decade, the importance of distributed services has become clear, as organizations have deployed networks to automate their business processes. As a result, organizations now depend on their networks and the related services, and any minute of down-time costs money. Not only personnel cannot perform their work during down-time, but many automated services, such as telesales, for instance, stop working. If an organization monitors its network, it can detect problems before they become a problem, increasing productivity and reducing personnel costs. This article provides a brief explanation of the SNMP protocol and explains the SNMP functionality that is available through PHP.
Simple Network Management Protocol
T
he Simple Network Management Protocol (SNMP) provides a framework for network management. SNMP enables access to network management information to make, for instance, status monitoring, faultdetection and configuration possible. Since its introduction, SNMP has been used heavily to perform monitoring and fault detection tasks. There has been little use of SNMP for configuration purposes, though, since it lacks a proper security protocol. However, the latest SNMP framework (version 3) defines security functionality by means of authentication, privacy and access control. These features protect the network against threats such as: 1. Modification of an SNMP message in transit by an unauthorized SNMP entity. 2. Modification of the SNMP message stream by changing the order, delaying or replaying messages. 3. Masquerading of SNMP entities through the appropriation of another SNMP entity’s identity. 4. Disclosure of management information that is December 2002 · PHP Architect · www.phparch.com
transferred between SNMP entities (users). 5. Eavesdroppers that attempt to capture the management data for other “evil” purposes. 6. Unauthorized access to particular management information. If you compare these capabilities with earlier version of SNMP, which only offered a single clear text password authentication method and no encryption, you’ll see that SNMPv3 represents a big step forward for the protocol’s security features. What’s more, SNMPv3 defines three different security levels: 1) No authentication and no privacy 2) Authentication and no privacy 3) Authentication and privacy It also allows for the use of different authentication and privacy protocols, thus enabling SNMP to evolve over time as new and better security mechanisms are REQUIREMENTS PHP Version: PHP 4 (4.3.0 for SNMPv3 support) O/S: Any Web server: Any Support Tools: N/A
25
FEATURES
SNMP Management With PHP Figure 1
defined. The SNMP framework
An SNMP manager requests management information from an SNMP agent. The manager invokes an SNMP request on an agent and the agent collects the requested information from its MIB and produces a response for the manager. The manager interprets the returned information and can use it for logging, status reporting or for automated actions. like initiating alarms. A typical example of this would be a polling system in which periodically traffic counters are collected by the SNMP manager.
Figure 2
An SNMP agent initiates a notification to a manager. Notifications are used when an agent detects an unusual or anomalous situation on its own, such as when an interface on the host goes down. Upon detecting the fault, the SNMP agent informs directly the SNMP manager of it, so that the latter can take the appropriate action, such as initiate an alarm.
Figure 3
December 2002 · PHP Architect · www.phparch.com
SNMP is built around an asymmetrical framework of many SNMP agents and one (or a few) SNMP managers. An SNMP agent resides in a remote host or device and is connected to the actual hardware that needs to be monitored. It provides access to the hardware via managed objects through an interface known as Management Information Base (MIB). The agent responds to queries from managers that request information from its MIB. A manager can therefore create a complete and coherent picture of the network’s status by requesting specific management information from multiple agents. The Management Information Base The Management Information Base (MIB) contains the managed objects under management by an agent. Briefly explained, the MIB is a virtual information store composed of one or more MIB modules. Each MIB module implements a specific set of management information, which is itself organized in an ordered tree. Only the leaves of this tree can contain real data, and then only if the object/node is defined as an ‘OBJECTTYPE’ (we’ll get to that in more detail later). The intermediate nodes used for creating the hierarchy do not actually contain any information, but provide the tree with a logical structure. Figure 3 illustrates an MIB tree example that is a part of the SNMPv2-MIB as defined in RFC 1907 (which can be downloaded from http://www.ietf.org/rfc/rfc1907.txt, if you’re interested in reading the complete MIB module specification). The tree depicted in this snapshot contains three objects (sysName, sysContact and sysLocation) representing scalar variables, as well as two objects ordered in a table under the sysOREntry node. The table objects are ordered by column (sysORID and sysORDescr), while each row is uniquely identified by an index. The indexes used in Figure 3 are 2 and, therefore, the table contains 4 values (2 columns * 2 rows). Each node in the MIB tree has a human readable name and is ordered by number. A specific node is described by an Object Identifier that combines the numbers of the branches along the path starting from the root. To find a particular leaf in the MIB, both an exact and a next search can be performed. The exact search requires that the name of the leaf being searched for exactly match what is saved in the data store, while the next search returns the first lexical graphical higher leaf. Figure 3 depicts a sample walk. Each arrow indicates
26
FEATURES the “next” of a particular object. To indicate that there are no more “next” values in the MIB tree, an exception called ‘EndOfMIBView’ is used. Structure of Management Information The language in which MIB modules are written is called Structure of Management Information (SMI). SMI was originally based on the Abstract Syntax Notation 1 (ASN.1). Let’s take a look at a small portion of this language - just enough to understand the creation of an MIB tree. The entities defined in a tree must all conform to the following standard format:
‘::=’ ‘{‘ ‘}’
The is used as a human readable identification of the object/node in the MIB tree. The provides specific information of the object. In the excerpt, all three definitions are of the ‘OBJECTTYPE’ kind, which includes the syntax of the object, the type of access granted to it (read/write), its status and its description. The provides the hierarchy information. Typically it consists of a single name and a single number. The name indicates the parent node in the MIB tree and the number indicates the current position in the tree with respect to its ancestors. For example, in Figure 3, the definitions of ‘system’ has the number 1 and the parent node is ‘mib-2’. The ‘sysContact’ entity defines the contact information of the particular system on which the MIB is defined, as mentioned by its ‘DESCRIPTION’ parameter. Its parent node is ‘system’ and its own numerical identifier is 4. SNMP and PHP After having guided you through a brief introduction to SNMP and its management framework, it’s now time to talk about the functionality of the SNMP extension in PHP. The PHP extension has been part of PHP for a long time, and it was originally developed by Rasmus Lerdorf as the UCD-SNMP module. At that time it supported the cutting edge version of SNMP—version 1. Over time, minor modifications have been made to extension, mainly due to changes in the packages it depends on, and it is currently called NET-SNMP. Some 3 years ago, I wrote an SNMP extension that made it possible to take advantage of the security features that are part of SNMP version 3. This was never included in the main distribution for various reasons. However, ever since I returned to Europe and went to live in Italy, I started working on the extension again and have included the security features of SNMP version 3 in the main distribution for the next release of PHP. December 2002 · PHP Architect · www.phparch.com
SNMP Management With PHP As we have seen in the previous section, SNMP activities can really be divided in two kinds of operations: retrieving information on request from the manager’s side and initiating notifications from the agent’s. The PHP language only supports the first form. This is mostly for historical reasons, since the first versions of PHP were used as scripting tools for web-based systems in which a notification from the client would simply have been impossible to implement: PHP scripts in web pages are only executed upon a web client’s request, and this methodology obviously does not allow for a continuous listening for notifications. In addition, the probability that a notification be received exactly when a web page is requested is pretty much zero. Thus, the only SNMP-related commands available in PHP are management information-related. Generally they can be used in polling operations. PHP-SNMP functions The SNMP functions in PHP can be divided into two groups, helper functions and protocol functions. The protocol functions that we discuss here are those that exist in versions of PHP that have already been released, and are based on SNMPv1, as are the example scripts. However, the upcoming new release of PHP (4.3) will include the new SNMPv3 functions and will support the new protocol security features. These new functions are discussed later in this very article. Helper functions The helper functions (Figure 4) are used only to alter the format in which the values passed to and from the protocol functions are used. These functions are only for internal use and do not communicate to external SNMP agents. The only parameter that they accept is a boolean that indicates whether a particular feature is to be used or not. The example shown in Listing 1 illustrates a script in which the helper functions are used to change the output resulting from calls to the protocol functions. Their output is shown in Figure 5. Protocol functions The protocol functions (Figure 6), on the other hand, are used to communicate with the remote SNMP agents. They are all used for data retrieval, except for snmpset(), which can change management data in a remote SNMP agent. These functions almost all take the arguments shown in Figure 7 Listing 2 provides an example of how these functions can be used in a real-life scenario. I have commented the script in detail to make it easier for you to follow it. SNMP In Practice Now that we have introduced all of PHP’s SNMP
27
FEATURES
SNMP Management With PHP Listing 1
1
functionality, we’ll take a look at a few examples to show how scripts that allow polling of network management data from a remote SNMP agent can be written. For our first example, we want to create a script that provides generic system data about the remote server, such as its hostname (sysName), its location (sysLocation) and the person responsible (sysContact) for its administration. These are the managed objects that we have used earlier to explain the MIB tree and the MIB module syntax. Listing 3 contains two approaches that achieve this result in two different ways. The first method consists of a series of explicit calls to snmpget() to retrieve the required information from the remote SNMP agent. This results in 3 requests on the wire. The second approach, on the other hand, is based on an MIB branch retrieval and subsequent filtering of the information needed. This last method results in a high number of requests being made to the agent, and does not represent the optimal solution to the problem, since the entire information tree is retrieved, even though only three of its values are actually needed. The output of Listing 1 is shown in Figure 8. The Interface table The second example, shown in Listing 4, retrieves some columns from a table and demonstrates how Figure 4: Helper Functions bool snmp_get_quick_print (void) void snmp_set_quick_print (bool quick_print)
indexes are used. The table whose values are being outputted is the interface table, or “ifTable”, which is December 2002 · PHP Architect · www.phparch.com
defined in the IF-MIB module published in RFC 2863 (available at http://www.ietf.org/rfc/rfc2863.txt). This table maintains information about the interfaces in a system; an interface is defined as an entry point to the next underlying layer in a protocol stack. A common example of this is the TCP/IP stack in network interfaces and serial line connectors. The New SNMPv3 Functions Recently, I have added new functionality to the SNMP module so that it can support the SNMP version 3 specifications and, in particular, its security model. All the protocol functions available in the SNMPv1 module have an equivalent in SNMPv3. The only difference between the new functions and the old ones is that their ‘snmp’ prefix must be changed into ‘snmpv3’. Also, the community password string used for SNMPv1 must be replaced by the parameters shown in Figure 9. As you can see in Figure 10, upgrading current scripts to SNMPv3 is very easy—all you have to do is change the function names and introduce the new security parameters. The PHP team took this approach to encourage all developers to upgrade their scripts and deploy the new security features for the management of their systems. The next release of PHP will also contain two new SNMP helper functions shown in figure 11. I introduced them as solutions to problems I encountered trying to interpret object identifiers and enumeration values in my scripts. The first function, snmp_set_enum_print(), causes the module to manipulate the data produced by SNMP agents so that, if it is part of an enumeration, its “human readable” value will be returned rather than its numeric one. The second, snmp_set_numeric_print(), causes values
28
FEATURES
SNMP Management With PHP
Figure 5 sysName (quickprint default/’’):STRING:orville.cardano.lisanza.net sysName (quickprint ‘1’) :orville.cardano.lisanza.net sysName (quickprint ‘’) :STRING: orville.cardano.lisanza.netharrie@orville$
Listing 2 1
December 2002 · PHP Architect · www.phparch.com
29
FEATURES
SNMP Management With PHP Figure 6: Syntax Protocol Functions
string snmpget (string target, string community, string object_id [, int timeout [, int retries]]) array snmprealwalk (string target, string community, string object_id [, int timeout [,int retries]]) bool snmpset (string target, string community, string object_id, string type, mixed value [, int timeout [, int retries]]) array snmpwalk (string target, string community, string object_id [, int timeout [, int retries]]) array snmpwalkoid (string target, string community, string object_id [, int timeout [, int retries]])
returned by a SNMP agent in numeric format not to be converted into human readable format. Numeric values are probably less intuitive to use for developers, but they possess the great advantage of remaining constant Figure 8 Explicit requests (SNMP-GET) The system name is: STRING: orville.cardano.lisanza.net The system administrator is: STRING: <[email protected]> The system location is: STRING: Cardano,Italy Tree retrieval, value selection (SNMP-WALK) The system name is: STRING: orville.cardano.lisanza.net The system administrator is: STRING: <[email protected]> The system location is: STRING: Cardano,Italy
and not being subject to change when moving from one system to another. Tips and tricks Many people dislike SNMP mainly because the do not understand how to read MIB modules or because they claim to have too little control over the SNMP
agent. Here are a couple of tricks that can make your life as a script developer a bit easier. 1) Determine first what you want to monitor from a high-level perspective and what is important for your deployment. This provides you as a developer with better guidance and direction and helps not to get lost in the SNMP details. 2) Use tools like libsmi (also open-source, and available from http://www.ibr.cs.tu-bs.de/projects/libsmi) that can help you by building an MIB tree for your viewing pleasure. If you use the ‘-f tree’ option, smidump will create a summary representation of the MIB modules that are part of a particular tree. This will enable you to determine the complete identifier of any object quickly and efficiently. 3) Use snmprealwalk() on your target agent as a tool to help you determine the identifiers of the objects you need. If you print out the results of a snmprealwalk() call, you can see what the human-readable form of a particular object identifier is. 4) Although this requires a bit more detailed knowledge of SNMP and MIB modules, use the numeric form of the object identifiers and the new snmp_set_numeric_print() function to retrieve the numeric form of your object identifiers. The advan-
Figure 7
string target
The target is used to define the hostname or the IP address of the remote SNMP agent as well to indicate the port (default 161) number the SNMP agent listens. Some examples are: “hostname” or “hostname:1661” where the first uses the default port number and the last the port number 1661 for the SNMP agent on the system ‘hostname’.
string community
The community string is an identification of the user/system that wants to access management information in an SNMP agent.
string object_id
The object identifier indicates the management information that is requested from the SNMP agent.
int timeout
The time out after which the SNMP request is resend if no reply is received.
int retries
The amount of retries that must be made before giving up.
string type and mixed value (only used for the snmpset)
The type and the value to which the object identifier in the remote SNMP agent will be set.
December 2002 · PHP Architect · www.phparch.com
30
FEATURES
SNMP Management With PHP Listing 3
1
Figure 9 string username
The username identifies who wants access a remote SNMP agent. This could be compare to the old community string of SNMPv1/v2C. The security level defines the grade of security and only the following levels exist:
string sec_level
- ‘noAuthNoPriv’ or ‘nanp’; a security level that is similar to community-based security. This level does not require any of the authentication and privacy parameters to be set and therefore must be provided as “” empty strings. - 'authPriv' or 'ap'; the highest security level that provides authentication and encryption. This level does require both the authentication and the privacy parameters to be set.
string auth_protocol
The authentication protocol defines the sort of authentication used. The following protocols are supported 'MD5' and 'SHA'
string auth_passphrase
The authentication passphrase is used to generate an authentication key with which the authentication is done.
string priv_passphrase
The privacy protocol defines the encryption protocol and the following protocol is currently only supported ‘DES’.
string priv_passphrase
The privacy passphrase is a secret phrase to be used to generate an encryption key.
December 2002 · PHP Architect · www.phparch.com
31
FEATURES
SNMP Management With PHP Listing 4
1 \n"; 47 echo "\n"; 48 echo "Interface | \n"; 49 echo "Description | \n"; 50 echo "Type | \n"; 51 echo "
"; 52 for (reset($ifIndex); $oid = key($ifIndex) ; next($ifIndex)) { 53 $i = $ifIndex[ $oid ]; 54 echo "\n"; 55 echo "" , $i , " | "; 56 echo "" , $ifDescr[$i] , " | "; 57 echo "" , $ifType[$i] , " | "; 58 echo "
\n"; 59 } 60 echo "\n"; 61 ?>
tage of using this approach is that numeric identifiers are unique in the MIB tree, and they never change— human readable forms may be modified. 5) Search for and read RFCs and books on SNMP. For instance, a good book with more in-depth analysis of the MIB modules is “Understanding SNMP MIBs” by D. Perkins and E. McGinnis, et al. (published by Prentice
December 2002 · PHP Architect · www.phparch.com
Hall), while a good book for the SNMP protocol is “SNMPv1, SNMPv2c, SNMPv3 and RMON”, written by W. Stallings and published by Addison-Wesley. 6) Last but not least, don’t be frustrated if you don’t get everything working right away. Take your time and work through your problems one at a time.
32
FEATURES
SNMP Management With PHP Figure 10: Protocol Functions SNMPv3
string snmpv3get ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) array snmpv3realwalk ( string host, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) bool snmpv3set ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id, string type, mixed value [, int timeout [, int retries]]) array snmpv3walk ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) array snmpv3walkoid ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]])
Signing Off Now that we have briefly introduced SNMP its related PHP functionality, I think that it’s fair to say that PHP is a simple and rich language for creating SNMP scripts. Due to the wide variety of extensions available for PHP, such as those that allow for database and graphics functionality, the potential is there to write complex management applications that provide very advanced capabilities. In fact, there are already two SNMP opensource packages available at www.sourceforge.net— Figure 11 void snmp_set_enum_print(bool enum_print) void snmp_set_numeric_print(bool numeric_print)
Just For Fun Network Management and PHP Network Analyzer—that make complex network analysis and management with PHP a reality. php|a
Harrie Hazewinkel is a long time participant in various open-source projects. He is the author of MOD-SNMP—an SNMP module with which the status of Apache can be monitored. He is an active participant in the Internet Engineering Task Force (IETF) and is the editor of the standard that defines managed objects for WWW services. His main expertise is Network Management and World Wide Web applications. Currently, he is an independent consultant and is open for new Internet challenges and adventures. You can reach him at [email protected].
www.cyberbite.com
CYBERBITE W W E E B B H H O O S S T T II N N G G Revolving around YOUR business Designed for PHP Programmers
Virtual Private Servers & Dedicated Servers
Reliable Internet hosting solutions
December 2002 · PHP Architect · www.phparch.com
Guaranteed 99.95% uptime
33
FEATURES
FEATURES
A Concrete Look at Database Abstraction Layers By Maxim Maletsky What is all this Database Abstraction fever? Why do we make so much noise about it and create so many different solutions, none of which seem to be able to make everyone happy? Perhaps, because not everyone tried to understand what the actual goal for abstracting database access is.
E
ver since PHP came on the market, proclaiming itself to be the world’s most beloved programming language, there has been a lot of disappointment over its weak cross-database portability. Indeed, for many developers, supporting pretty much all the existing databases through individual modules was simply not enough. Developers also want the possibility to avoid hardcoding db-specific functions in their applications— they want to call only one module (or object), tell it what DBMS to use and access their data transparently. As it turns out, that wasn’t a reasonable addition to PHP—simply because of its nature. First of all, PHP is not a true object-oriented language—it uses procedural functions to interact with DBMS systems, and libraries for different databases have to be called in different ways. Second, DBMS extensions are, internally, independent of each other—it would thus be almost impossible to synchronize them into one single object without first creating an OOP wrapper, which brings us back to the initial problem. PHPLIB As PHP evolved, PHPLIB came into existence. PHPLIB was a PHP library whose main goal was to allow for session management in PHP3. Session information was saved in a database and, with the intent of providing cross-database functionality, a transparent layer was used for data access. Not only you could save sessions into your favorite DBMS, but there was also a way for
December 2002 · PHP Architect · www.phparch.com
you to directly access these databases from your scripts, thus making PHPLIB the very first data abstraction layer available to PHP users. In PHPLIB, you’d call your database with something like this: This example would print an HTML table filled with 1 query("select * from employees"); 6 7 $t = new Table; 8 $t->heading = "on"; 9 $t->show_result($db); 10 ?>
results. Handy, isn’t it? Before you could get away with accessing your database using just five lines of code, however, you’d have to specify what DBMS should have been used via an included file. That was convenient for most cases—particularly for projects where the DBMS could have changed at some point in the future. If that “lucky event” were to happen to you, you’d simply have to REQUIREMENTS PHP Version: 4.0 and up O/S: Any Web server: Any Support Tools: A Database System supported by PHP
34
FEATURES
A Concrete Look at Database Abstraction Layers
edit the main PHPLIB configuration file to point to a new database configuration file. Besides sessions and DB functionality, PHPLIB also provided you with lots of other goodies, such as an authentication system, HTML rendering classes and much more. Unfortunately for PHPLIB (and luckily for all of us), PHP4 came out with sessions incorporated by default, making them simpler to use and allowing for the full cross-platform compatibility with no need to use any database at all that everybody was waiting for. When that happened, PHPLIB started becoming less and less popular—to the point that it was used just for its DB abstraction capabilities and, of course, the other “goodies” that people were so addicted to. Alas, even that didn’t last for too long... PEAR Soon after PHP4 was released, the PEAR project began. PEAR is a library similar to PHPLIB, with the allimportant exception that it was developed officially along the PHP core itself. It contains an extremely powerful set of database abstraction functionality—as well as many more useful tools. There are dozens of people who are working on PEAR today and thousands of people who use it. One of the main goals of PEAR is to provide PHP with a flexible database access system available on any platform (and for any DBMS). An example of database use with PEAR is shown in Figure 1. The PEAR DB class makes it quite easy to work with multiple databases. PEAR also allows you to use Data Objects so you can build very dynamic queries and connect several of the other available PEAR components to it. For instance, if you wanted to create a query which selects all the employees who are between 18 and 35 years old, with Data Objects you could dynamically build it this way:
1 whereAdd(‘age > 18’); 5 $employee->whereAdd(‘age < 35’); 6 $employee->find(); 7 8 while ($employee->fetch()) { 9 echo "{$employee->id} {$employee->name}
"; 10 } 11 ?>
This would compose and internally execute the following query:
SELECT * FROM employees WHERE age > 12 AND age < 30
December 2002 · PHP Architect · www.phparch.com
More DB Abstraction Modules Besides PHPLIB and PEAR, there are also a couple of other built-in extensions to PHP that allow for database abstraction, such as DBA and DBX. They are fast and flexible, but somewhat lacking in the compatibility department. There is also ADODB—a great and very powerful database library that is now available on a number of platforms. What the Ideal Database Abstraction Layer Should Do All these Database Abstraction Layers miss (at least) one thing. Sure, they do let you switch easily between all kinds of DBMS systems. The problem is that database systems do not simply differ in the way PHP interacts with them: they also often use a particular “dialect” of the SQL language that differs from the standard in subtle but significant ways. Thus, even though, theoretically, the word “Abstraction” between “Database” and “Layer” means you can accomplish the same task on any database server without editing a single line of your code, this is not entirely possible today because you will still have to modify your queries. For example, assume for a moment that we need to find out my age through SQL having only a string containing my birthday. In MySQL, the code will look like this: SELECT ( YEAR ( NOW() ) YEAR ( ‘1978-10-26 20:38:40’ ) ) as age;
In PostgreSQL, however, that will not work, and will have to be changed as follows:
SELECT ( EXTRACT( YEAR FROM TIMESTAMP ‘now’ ) EXTRACT( YEAR FROM TIMESTAMP ‘1978-10-26 20:38:40’ ) ) as age;
Oracle, on the other hand, requires quite a bit more work, as shown in Figure 2. All three SQL databases I used in the example claim to fully support SQL standards. The problem is, unfortunately, that the SQL standard is somewhat limited and not always adequate to the actual needs of every developer—as a result, you’ve just had quick proof that most DBMS systems do not (and, likely, never will) share the same SQL standard at all. Thus, if you’re used to having your SQL queries hardcoded in your applica-
35
FEATURES
A Concrete Look at Database Abstraction Layers
tion, you are going to have a very hard time rewriting most of them if you need to migrate it from one database to another. Ideally, a good Database Abstraction Layer should support differences in the SQL dialects for you. One way of doing this would be to dynamically rewrite the SQL code right within the db layer (such as ADODB does in some cases). However, as you can already imagine, this isn’t always a good solution, even if many DB layers try to implement it. For one thing, it’s not always possible to convert a SQL string written for a particular DBMS into its equivalent for a different DBMS in a univocal way; this means that it would be difficult (and dangerous) for the DB layer to “guess” what a SQL query means in every possible case. Another reason, and perhaps the most crucial one, is that parsing, “rethinking” and “repairing” every single SQL command would cause a significant performance hit that is not easily justified. The “good” database abstraction layers that we have today already slow things down enough—even without this feature. Finding a solution to this problem may look like an impossible dream. In reality, all it takes is a bit of foresight and some level of discipline. All you have to is to collect and save your SQL queries in a central location and use a sound dynamic mechanism to load them into
Theoretically, the word “Abstraction” between “Database” and “Layer” means you can accomplish the same task on any database server without editing a single line of your code. your code as needed. Consider the code in Listing 1—let’s call it mysql.sql.php—in which you hold your queries. As you can see, a separate variable is required for each SQL statement. Its value will later be injected into a call to a database, resulting in a system that can easily adapt to any DBMS—all you would have to do is to create another file, say postgresql.sql.php, that contains the SQL syntax appropriate for the DBMS you want to use. Of course, you will need a little framework that can use these files, providing you at the same time with the
rapX, don’t R.I.P. “Dearly Beloved, we are gathered here today to mark the passing of a fine young software product. So rich in promise, so beloved by its users, so carefully tended by its developers...”
rap-x is the collaboration platform that enables software developers to generate revenue from their existing users. Click for a demo at rap-X.com, or email [email protected] for more information
December 2002 · PHP Architect · www.phparch.com
36
FEATURES
A Concrete Look at Database Abstraction Layers Listing 1: mysql.sql.php
1
ability to call the appropriate PHP functionality for the DBMS you choose to use. To put it simply, you need to come up with a simplified (but more powerful) version of what PHPLIB and PEAR do. Luckily, PHP supports Object-Oriented Programming (OOP) to a degree that is going to make this possible, and classes adapt very Figure 1 1 query($sql); 7 while ($row = $result->fetchRow()) { 8 echo $row[2] . '
'; 9 } 10 $db->disconnect(); 11 12 ?>
Figure 2 SELECT TO_CHAR ( TO_CHAR (sysdate,'YYYY') TO_CHAR ( TO_DATE ( '1978-10-26 20:38:40', 'YYYY MM DD HH24:MI:SS' ) , YYYY' ) ) "age"
well to this kind of requirement. There will be no actual queries passed to the class— we’ll pass a set of “aliases” (which we assign in mysql.sql.php) instead. This way, we will later be able to modify the SQL statements at will to adapt them to any DBMS from a single location. The first step consists of creating a “base” class that defines our fundamentals—the functions for connecting to a database, executing queries managing recordsets, and so on. This is shown in Listing 2, and provides us with a foundation of database-independent code to work on. Next, we need to create a specialized layer that sits atop the db_layer class to provide specialized database functionality. mysql_class.php (included in this month’s ZIP file), for example, shows a version of a db_layer subclass that provides the functions required to use MySQL. The mysql class implements those functions that the db_layer code calls whenever db-specific functionality is required. Under normal circumstances, a PHP script (such as the one in Listing 3) will need to instantiate our mysql class and then just call the “generic” functions defined by its db_layer superclass to access the database. Naturally, you can add more functions to the class (for example if you want to implement transactions), but you will always have to implement them using the same two-step approach, so that they can be adapted to other, different DBMS systems. A Look Under the Hood
FROM dual;
December 2002 · PHP Architect · www.phparch.com
Let’s now take a look at how the classes work. As you can notice within the code, most of the magic is per-
37
FEATURES
Figure 3 1 connect('username', 'password', 'dbname', 6 'my.host.com'); 7 8 /* Pass alway same input values */ 9 $result = $db->query('age', 10 Array( 'date' => '1978-10-26 20:38:40' )); 11 12 /* And retrieve always same results */ 13 print_r($result); 14 ?>
formed by the db_layer class in response to the value of the $this->api_name variable, which it uses to determine where the SQL commands should be taken from (in our case, the mysql.sql.php include file). Each of the other db_layer functions are simply used as wrappers to the real database access calls that take place inside the appropriate subclass. What if you needed to change your DBMS? The first step would consist of creating another subclass of db_layer that implements the calls specific to your new database system. Next, you would need to create a new include file where all the queries are defined and, finally, you would need to change your PHP code so that it instantiates your new class rather than the old one. For this reason, I recommend that you instantiate
Listing 2 1 db_username = $user; 35 $this->db_password = $pass; 36 $this->db_database = $db; 37 $this->db_host = $host; 38 39 // Return positively on successful connection. 40 Return $this->api_connect(); 41 } 42 43 44 /* Method: query 45 +------------------------------------------------------------------+ 46 Description: Execute Query 47 +------------------------------------------------------------------+ 48 */ 49 50 function query($alias = '', $data = '') { 1
Continued On Page 40...
December 2002 · PHP Architect · www.phparch.com
38
FEATURES the database class in a global include file—so that you would only need to change it in one place should you ever need to switch DBMS. A Few Closing Notes As you can imagine, it’s easy to come up with a number of possible extensions to the code I presented in this article. One possibility is the use of transactions, as I mentioned above. Another one could be the addition of some specialized form of error handling, which would give you the opportunity to trap errors and provide “sanitized” notification messages to your users. Similarly, you could create specific configuration instructions for every DBMS right within their respective SQL files. This could be handy when one database has to do something that others can’t or don’t need to. For instance, I used this method in a situation in which a single Oracle query was worth three of MySQL’s, and yet I needed to make a single call and obtain the same result set. To update two tables with Oracle, I used a PL/SQL code block or a stored procedure, both unavailable in mySQL. This would make a single codebase impossible, since in one case you’d have to execute a single command, while in the other several SQL statements have to be run. If you were to modify your abstraction class so that multiple queries can be specified with a single alias, your call would still look like the code in figure 3. Handling Database Abstraction this way gives you infinite possibilities (as well as infinite possibility to
needlessly complicate things). The trick is in only using this approach if you think that there will be a reasonable need in the future to move your current applications to a different DBMS. However, Database Abstraction can be a very powerful tool, particularly if you’re writing applications that will be used by different people in different environments, as it is often the case with open-source projects. Think how complicated would be to write a forum application and then distribute it over the world without having a well-structured DB abstraction layer! To see a more complex, yet fully working example of Database Abstraction logic similar to the one I used in this article, take a look at the ZoomStats project on S o u r c e F o r g e (http://www.sourceforge.net/projects/zoomstats). It was born as a web traffic analyzer, but is slowly becoming a whole OOP Database Abstraction Layer of its own! Also, both PHPLIB and PEAR are very much alive and kicking; you can find the former at http://phplib.sourceforge.net and the latter at http://pear.php.net. php|a
Maxim Maletsky is an IT Consultant currently taking care of the e-government infrastructures in Italy. Maxim’s spare time and sleep are almost completely wasted on hassling with the official Oracle extension for PHP, bug hunting, phpnotes moderation, answering all kinds of posts on PHP mailing lists and writing. You can get in touch with him at [email protected].
Listing 3 1 connect('username', 'password', 'dbname', 'my.host.com'); 16 17 /* Here you will execute the query and get 18 * the result set returned as array 19 */ 20 21 $result = $db->query('age', Array('date' => '1978-10-26 20:38:40')); 22 23 print_r($result); 24 25 /* This should print you: 26 * Array 27 * ( 28 * [age] => 24 29 * ) 30 */ 31 ?>
December 2002 · PHP Architect · www.phparch.com
39
FEATURES
A Concrete Look at Database Abstraction Layers
Listing 2: Continued From Page 38... 51 52 // Connect to API 53 $this->connect(); 54 55 // Store Alias 56 $this->db_alias = $alias; 57 $this->db_data = $data; 58 59 extract($this->db_data); 60 61 // Include SQL storage files 62 @include($this->api_name . '.sql.php'); 63 64 // Check existence of query 65 $this->db_query = isset($query)? $query : die('No query for alias ' . $this>db_alias); 66 67 // Clean trailing semicolons ';' in SQL 68 $this->db_query['sql'] = isset($this>db_query['sql'])? preg_replace("/;?\s*$/s", '', $this->db_query['sql']) : ''; 69 70 // Execute Query 71 $this->api_execute(); 72 73 // Return result: 74 Return $this->result(); 75 } 76 77 78 /* Method: result 79 +------------------------------------------------------------------+ 80 Description: Return results. 81 +------------------------------------------------------------------+ 82 */ 83 84 function result() { 85 86 // Number of Rows 87 $this->db_rows = $this->api_rows(); 88 89 // Number of Columns 90 $this->db_cols = $this->api_cols(); 91 92 // Compose array 93 $r = 0; 94 while($r<$this->db_rows and $this->api_next_record($r)) { 95 96 // Loop values 97 foreach($this->db_record as $col=>$val) { 98 $this->db_result[$r][$col] = $val; 99 } 100 101 // increment row count 102 $r++; 103 } 104 105 // Return Results 106 Return $this->db_result; 107 } 108 109 /* Method: close 110 +------------------------------------------------------------------+ 111 Description: Close Database Connection 112 +------------------------------------------------------------------+ 113 */ 114 115 function close() { 116 117 // Close database connection 118 $this->api_close(); 119 Return True;
December 2002 · PHP Architect · www.phparch.com
40
FEATURES
FEATURES
Creating A Multi-iitem Visual Catalogue System With PHP & Internet Explorer By Davor Pleskina Many web-based “Catalogue Systems” have cropped up on the Internet scene lately, either as online stores or advanced ordering systems of some kind (such as subscription systems). The catchwords that are usually employed to describe them include “interactive” and “visual”—which usually means simply that users (or, should we say, potential clients) visiting them should be able to review the products offered for sale through sets of images, add them to a virtual shopping basket through a single mouse click and, finally, let the system process their order.
C
learly, this entire process should be as seamless as possible: users don’t need to know how the system works, and any hiccups can cause the store owner to lose a customer. A Look at How a Catalogue System Works
As with many computer-related technologies, there is no real “standard” way of developing a “visual” catalogue system. However, most of the existing implementations seem to have focused on either showing pic-
We will always run the same script, but it will behave in a different way, depending on the parameters that are passed to it. tures of individual products, or showing pictures in which more than one product is displayed (as it is often the case, for example, with stores that sell spare parts). In that case, each of the products in the picture can be clicked on individually.
December 2002 · PHP Architect · www.phparch.com
Whenever confronting the need to develop a new catalogue system, developers tend to get lost in technology—names like Macromedia Flash and Java start to float around and, pretty soon, a simple idea becomes a complex monster. In reality, it is possible to develop a very powerful catalogue system using mostly serverside functionality, with just a little help on the client side. In this article, I present a simple and yet powerful “visual” catalogue system, based on PHP and MySQL for server-side functionality, and nothing more than a lot of IE HTML and some Javascript on the client side. The great thing about a system like the one I will show here is that it requires very little in the way of functionality as far as the browser is concerned. This, in turn, means that there won’t be a need for any plug-ins or other downloads and that your application will be compatible with a great majority of the client systems out there. Best of all, our entire catalogue system will require just one file in the frontend—thus providing a compact profile that can be optimized to maximum extent possible.
REQUIREMENTS PHP Version: PHP 4 with Sessions enabled O/S: Any Web server: Any Support Tools: MySQL
41
FEATURES
Visual Catalogue System With PHP & IE
Scope of The Application Our system will support the following functionality: · Catalogue Selection · Article Selection · Basket Preview Naturally, a complete on-line catalogue system would normally support more functions than what we have here, but they would be outside the scope of this article. These include basket editing, order processing, Figure 1
handling more than one product set, and so forth. Most of these functions are just a small step away from the ones we will look at here and can be easily added through techniques similar to the ones that I have used. Our sample application will only provide two “catalogues”, each represented by a single JPEG image that contains a variable number of products. Users will be first asked to select a catalogue, and then a product by clicking on it in the image. Finally, they can specify a number of items for that product and add them to their shopping basket. At any time, users can also switch to a different catalogue or review their baskets (unless, of course, the application has just started, in which case a catalogue must be selected before any other action can be undertaken). Figure 1 shows the system’s workflow diagram. In “Catalogue Selection” mode, each catalogue is presented as a link that can be clicked to show the products it contains. When a user clicks on a catalogue, December 2002 · PHP Architect · www.phparch.com
the system immediately goes into “Article Selection” mode, showing the catalogue’s image with clickable areas for each of the products. Moving the cursor over these areas displays a simple “tooltip” containing information about the appropriate product. Once the user clicks on one of the areas, the system shows an input form with a “Quantity” field and an “Add” button for the selected product. Pressing the Add button stores the product and quantity information into the basket and displays a confirmation message, returning at the same time to Article Selection mode. The system also features a “Basket Preview” mode, which shows the basket’s contents. Sounds simple? Well, the process is really not that complicated at all—it would have to be that way, since the goal of the application is to make it as easy as possible for the user to make a purchase. To make everything work in a smooth and optimized way, however, we’ll have to dig a few tricks out of our good old PHP hat. So, How Are We Going To Do That? Let’s take a look at some of the techniques that we’ll use to develop our visual catalogue. First of all, since it’s “visual”, we’ll need a few pictures—I’ve added a couple of them in the images/ folder of the project (you should find all the code in the ZIP file that you received when php|a was delivered to you). We’ll be selling two types of products (and, therefore, we’ll use two catalogues): fruits and kitchen accessories. Next, we’ll need someplace to store the information about our catalogue and, since we’ve mentioned the use of MySQL, we’ll store all our data in Figure 2 a set of nicely defined tables (shown in Figure 2). You’ll notice that, because we use a single image for each catalogue, we’ll also have to describe the areas that correspond to each product in the catalogue. This is done in the CATAREA table, which also links articles to catalogues. Naturally, our tables need to be filled with some data, which you can see in Figure 3. The CATAREA table is used as a “bridge” between CATALOGUE and ARTICLE, so that each article can be assigned to the proper catalogue. In addition, the POINTS column is also used to specify the bounding areas that define the position of each article in its
42
FEATURES
Visual Catalogue System With PHP & IE Figure 3
Table “CATALOGUE”: CAT_ID DESCRIPTION ——— ————————————————————1 Fruit and Vegetables 2 Accessories
IMAGE_NAME ———————————— cat1.jpg cat2.jpg
Table “ARTICLE”: ARTICLE_ID ————— APPLE PAPRIKA LEMON ONION MANDARINE SUGARPOT PLANT
ARTICLE_NAME —————————Apple Green paprika Fresh lemon Onion Mandarine Sugar pot Plant in the glass
HONEY
Honey with dry fruit
DESCRIPTION —————————————————This is tasty and very sweet apple! Have you ever tried fried paprika? Make yourself a lemonade! Excellent salad add-on! Enjoy this sweet fruit Use it while drinking coffee Decorate your working space with this plant Delicatessen
PRICE ——5.00 3.20 4.00 2.10 3.50 11.45 16.00 20.00
Table “CATAREA”: CAT_ID ————1
AREA_ID ———1
ARTICLE_ID ———— APPLE
1
2
LEMON
1
3
MANDARINE
1
4
ONION
1
5
PAPRIKA
2
1
SUGARPOT
2
2
PLANT
2
3
HONEY
POINTS —————————————————————— 421,321,463,265,536,254,594,279,636,353, 611,425,524,454,470,436,420,375 57,192,68,120,147,69,226,95,215,165,146,216, 77,220 15,315,40,266,99,245,150,255,191,311, 178,363,156,392,104,411,51,398,28,362 433,181,456,138,486,123,479,39,532,39, 525,113,568,143,595,196,572,238,515,255, 478,260,442,215 238,327,240,136,308,103,382,129,401,192, 398,248,356,356,333,379,298,392,272,383 121,125,187,198,228,287,215,334,158,366, 98,351,70,301,86,244,82,195,98,138 182,178,217,90,434,89,481,184,407,216, 385,272,330,292,271,267,253,214 457,206,533,193,591,225,595,284,567,353, 492,391,452,372,425,298
respective catalogue image. These, in turn, will be used later on by our PHP script to create an HTML MAP element so that the user can actually click on the articles and produce the desired result. Another important technique we are going to use is sometimes referred to as “Parametrized Script Navigation”. Fancy words aside, this simply means that we will always run the same script, but that it will behave in a different way, depending on the parameters that are passed to it when the browser is requesting a page. The advantage of this method is that all your code is contained in a very limited number of files (only one in our case!), and that promotes reusability. The disadvantage is that it’s easy to let things get out of hand and end up with a bowl of spaghetti. With a bit of care, however, this problem can be easily taken care of. We will call our navigational parameters “Mode” and
December 2002 · PHP Architect · www.phparch.com
“Action”. The former will represent the application’s current state (such as “Catalogue Selection”), while the latter will indicate what action should be taken under current Mode. I’m sure that some of you might be wondering, at this point, whether the two parameters could have been incorporated in a single GET variable. In this case, I felt that two separate parameters would make extending the application much easier in the future. Since the catalogue system that we’re looking at here is rather simple and must be expanded upon to build a complete application, I think that a modular design approach brings many benefits. On To The Code The first things that we are going to need in our script are some variables whose life spans beyond the
43
FEATURES
Visual Catalogue System With PHP & IE
script’s execution. These variables should be registered within a session. Sessions are PHP’s way to preserve certain data across subsequent calls to scripts on a web server or in a web server farm. Sessions start from the first access to the script and last until either they are explicitly destroyed or after a pre-defined period of inactivity. Session state is maintained either through a cookie or by appending a specific parameter to the script’s URL. We’ll use the former method, since it’s generally more discrete and convenient. A session should be started by calling the session_start() function before any contents (including blank lines) are outputted to the client. This is because the server must write a cookie to the browser, and this is done through the page headers. If your copy of PHP has output buffering turned on, this is not necessary, but you should adopt this methodology as a rule to maximize the portability of your code. Our session initialization code is shown in Listing 1. Looking closely, you will notice that we assign values to three variables, $mode, $action and $cat_id, which Figure 4
Change Catalogue Select Articles View Basket
These links never change and are always shown to the user (you can see them at the top of Figure 4). As you can see, each link redirects to the viscat.php script itself, and that the two parameters p_mode and p_action are passed to it. When the users click on any of these links, their browser will reopen the script but with different parameters, allowing our script to decide which code blocks to execute. Listing 2 shows how these parameters are used in our code. If no value is assigned to them, the script is probably being executed for the first time and $mode and $action will be set to “1” (Catalogue Selection) and “0” (Default Action) respectively. If at this point, for any reason (such as potentially malicious usage of the script), $cat_id is less than 1 we print out an error and force the user to choose a new catalogue before continuing: if (($mode > 1) && ($cat_id < 1)) {
respectively contain the current mode, the action to be performed and the ID of the catalogue currently selected. They are assigned different names from their URL parameter equivalents for a very specific reason. If you were to enable the register_globals functionality in your copy of PHP (it has been disabled by default since PHP 4.2), having your internal variables and the URL parameters share the same name could lead to severe security issues, since a malicious user would be able to manipulate the values internally used by your scripts. After we have processed our initial tasks and prepared our script for web page generation, it’s time to output a few links that will go in our header:
print “Please select a catalogue first!
”; $mode = 1; }
This way, we can we can avoid logical errors by ensuring that a catalogue has been chosen before undertaking any other actions. Enter the Database After this initial parameter checking, we must connect to our MySQL database in order to allow the rest
Listing 1 1 10
December 2002 · PHP Architect · www.phparch.com
44
FEATURES Listing 2 1 "; 13 print "MySQL Error " . mysql_errno() . ": " . mysql_error() . "
"; 14 print "Exiting..."; 15 // We leave... 16 exit; 17 } 18 19 } 20 21 22 ?>
of the script access to our dynamic catalogue data (Listing 3). The variable $mycon_id will be set to a valid MySQL resource in case of a successful connection. Otherwise, the script will output an error and stop its execution. In my example code, I assume that the database the application relies on is called “davor”, and can be accessed by the user “root” with no password. Naturally, in your own experiments you are free to use your own settings—just remember to change them in the source code. The last thing to do in this part of the code is to display the appropriate information about the active catalogue. If a catalogue was previously selected, the $cat_id variable must be set to a value greater than 0. The code in Listing 4 will extract its name from the CATALOGUE table and show it at the very beginning of our HTML table. We also store the name of the catalogue’s image in the $image_name variable.
Case In Point We have now reached the decisional part of our script, shown in Listing 5. The script uses the “switch” control structure to check the value of $mode and branch the script’s execution to specific functions designed to handle each of the three application modes. As a special case, let’s now take a look at what happens when the script is accessed for the first time. The $mode variable will be set to 1, thus forcing the system into catalogue selection mode. The function CatalogueProcedures() will be called, resulting in the display of the catalogue selection page. Clicking on any link except “Change Catalogue” in this state will result in the message “Please select a catalogue first!” being displayed and no further action being taken.
Listing 3 1
From here on, we will generate HTML code using PHP --> Let's connect to our database - we will use $mycon_id variable; If a connection has already been established, $mycon_id will be other than NULL... (!$mycon_id) $mycon_id = mysql_connect("localhost", "pleskina_public", "viscat"); if ( (!$mycon_id) || (!mysql_select_db("pleskina_viscat", $mycon_id)) ) { // Let's inform about the error and get out print "Could not connect to the database!
"; print "MySQL Error " . mysql_errno() . ": " . mysql_error() . "
"; print "Exiting..."; // We leave... exit; }
December 2002 · PHP Architect · www.phparch.com
45
FEATURES
Visual Catalogue System With PHP & IE Listing 4
1 0) 4 { 5 $data_row = mysql_fetch_array( 6 mysql_query( "SELECT * FROM CATALOGUE WHERE CAT_ID='" 7 . mysql_escape_string($_SESSION['cat_id']) . "'", $mycon_id) 8 ); 9 $image_name = $data_row["IMAGE_NAME"]; 10 print "<EM>Active catalogue is: " . $data_row["DESCRIPTION"] . "
\n"; 11 } 12 13 ?>
Choosing an Article: The Real Fun Begins So far, things have been quite simple—but they’re about to get complicated. In Article Selection mode, we must show the appropriate catalogue image and generate the image areas to allow the user to click on the item he or she is interested in. Let’s look at the code in Listing 6, which shows the source for the ArticleProcedures() function. Here, we have two possible actions, but we use a little trick because one of them (action 0) is really a portion of the other. While going through the case statement, the action zero commands will be performed even when $action is set to 1, thanks to the fact that the case 1 block does not end with a break; statement. Although it can seem illogical at first look, coding action 1 first is quite reasonable—this action means that some article was passed to the script and that it
should be added to the basket. Even if that is the case, we still have to provide the user with an opportunity to view and choose the other articles—and that is the duty of the case 0 code block. This works by outputting a <MAP> HTML code block called #imagemap, which is defined in the usemap parameter of the image tag for the current catalogue. The map contains the coordinates of selectable (clickable) polygons that define the shapes of each article on our catalogue image. Each AREA tag is represents a polygon whose coordinates are relative to the catalogue image and are retrieved from the CATAREA table. The alt tag provides name, price and description information that is displayed when the user moves the cursor over the polygon. A little Javascript magic allows us to transfer some of the catalogue’s functionality directly to the client. The onClick event added to each AREA tag passes two parameters, a text string and the value of ARTICLE_ID,
Listing 5 1 Catalogue Selection:
"; 10 // Now invoke procedures used to select catalogue etc. 11 CatalogueProcedures(); 12 break; 13 } 14 case 2: // Selecting Articles 15 { 16 print "Selecting Articles
"; 17 // Call procedures handling article selection 18 ArticleProcedures(); 19 break; 20 } 21 case 3: // Selecting Articles 22 { 23 print "Basket Preview
"; 24 // Go to function which handles basket contents 25 BasketProcedures(); 26 break; 27 } 28 } 29 30 ?>
December 2002 · PHP Architect · www.phparch.com
46
FEATURES
Visual Catalogue System With PHP & IE
to the script that is dynamically generated in lines that follow. The ShowArticleInfo() Javascript function changes HTML content of a element named “artinfo” so that it contains a