rarecore development

NFS, the unperformed Network File System

September 29th, 2008

As described, we are using 2 web servers and 1 application server. What’s the background? It’s easy to explain. Every good designed and developed website needs a template engine for visualization. Our template engine is smarty. Smarty has the caching feature and I like it to minimize the database connections and optimize the performance. I cache some pages (profile, news) until the next update (e.g. new comment) was done.

The problem is if the PHP script has deleted the cache file on the first web server (tbweb01), the cache file on the second web server (tbweb02) isn’t deleted. To avoid any delay times, we stored our source code on the application server and we are using NFS to share the files. Normally it works fine, but on some peak times the NFS is too slow.

Information like “logged in user” and so on are saved on the session. Usually the user opens the website and the browser always uses the same ip address. It does not matter which server it is. If the user uses a proxy, the proxy chooses one of the round robin ip addresses by each click. So on one server the user has a correct session with the information, on the other server he hasn’t. If we store the sessions on the NFS we make sure that every server uses the same session. But the performance…

Now we have to search for a good solution!
I found two interesting articles. One about a session_set_save_handler and the other about a cache_handler_func. And a new idea was born. Use NFS only to read the files and store the information in the database.

session_set_save_handler:
This default PHP function allows you to use your own way for the session handling. I think it’s really better to store everything in a mysql table and avoid the NFS storage. On the article they didn’t open and close a database connection. That’s bad… Normally your page is already using a connection, but sometimes you don’t have to open one.

It’s a little bit tricky, but not a problem for experienced developer. I wrote a class to handle my database connection(s). It was very important to use one class for more connections, but it was much more important that the class doesn’t connect with the same user and the same selected database twice times and make sure that the class closes every connection correct. I called my class “classDataBase” and overloaded the most used mysql functions. To make sure that the handling works, we’ve to use a static array for the connection and remember the used index in an attribute. Also we remember the count of objects for each connection. On calling the close function, we don’t want to close always the database connection. If more than one object uses this connection, we only decrease the count variable. To be sure that the PHP script closes all opened connections correctly, the class registers automatically on the initializing of the first object a static DBCloseAll() function for shutdown.

classDataBase.php

/******************************************
 * class created by Exi (http://www.technobase.fm/member/1)
 * please find some information about on
 * http://blog.rarecore.eu/nfs-the-unperformed-file-system.html
 ******************************************/

class classDataBase {

    public static $connections;
    var $connection;
    var $Idx = 0;
    var $DBName; 

    // error handling
    var $errNum; // error number
    var $errTxt; // error message

    // constructor - set default values
    function classDataBase() {
        if(!is_array(classDataBase::$connections)) {
            classDataBase::$connections = array();
            register_shutdown_function(array('classDataBase', 'DBCloseAll'));
        }

        $this->errNum = 0;
        $this->errTxt = "";
        $this->connectionID = false;
    }

    function Init($cDBName, $cDBHost, $cDBUser, $cDBPass) {
        $found = false;
        $retVal = false;

        if($this->connection) {
            $this->DBClose();
        }

        for($i=0; $i<count(classDataBase::$connections); $i++) {
            $a = classDataBase::$connections[$i];

            if ($a['dbname'] == $cDBName &&
                $a['dbhost'] == $cDBHost &&
                $a['dbuser'] == $cDBUser) {
                    if($a['connection']) {
                        $this->connection = $a['connection'];
                        classDataBase::$connections[$i]['count']++;
                        $this->Idx = $i;
                        $found = true;
                        $retVal = true;
                        break;
                    }
            }
        }

        if(!$found) {
            $a = array();

            $retVal = $this->DBConnecting($cDBName, $cDBHost, $cDBUser, $cDBPass);

            if(!$retVal) {
                //Lost connection to MySQL server at 'reading authorization packet', system error: 0
                if($this->errNum != 2013) {
                    $mail  = "Error on connect to the Databasen";
                    $mail .= "Error Text: ".$this->errTxt."n";
                    $mail .= "Error Number: ".$this->errNum."n";

                    $this->ErrorMail($mail);
                }

                echo "<strong><center>Error on connect to the Database</center></strong>";
                exit();
            }     

            $a['dbname'] = $cDBName;
            $a['dbhost'] = $cDBHost;
            $a['dbuser'] = $cDBUser;
            $a['connection'] = $this->connection;
            $a['count'] = 1;
            array_push(classDataBase::$connections, $a);
        }

        $this->DBName = $cDBName;

        return $retVal;
    }

    function DBConnecting($cDBName, $cDBHost, $cDBUser, $cDBPass) {
        $retVal = true;

        // connect to database
        $this->connection = @mysql_connect($cDBHost, $cDBUser, $cDBPass);

        // throw error if connection failed ...
        if (!$this->connection) {
            $this->errNum = @mysql_errno();
            $this->errTxt = @mysql_error();
            $retVal = false;
        } else if(!@mysql_select_db($cDBName, $this->connection)) {
            $this->errNum = @mysql_errno();
            $this->errTxt = @mysql_error();
            $retVal = false;
        }

        return $retVal;
    }

    //function to execute statements
    function execSQL($SQLQuery, $mail = true) {
        $XSQL = false;

        if($this->connection) {
            $XSQL = mysql_query($SQLQuery, $this->connection);

            // Check for error
            if (!$XSQL) {
                $this->errNum = @mysql_errno($this->connection);
                $this->errTxt = @mysql_error($this->connection);    

                if($mail) {
                    $this->LogDBError($SQLQuery);
                }
            }
        }

        return $XSQL;
    }

    function query($SQLQuery) {
        return $this->execSQL($SQLQuery);
    }

    function queryCatch($SQLQuery) {
        return $this->execSQL($SQLQuery, false);
    }

    function insertId() {
        return mysql_insert_id($this->connection);
    }

    function affectedRows() {
        return mysql_affected_rows($this->connection);
    }

    //function to fetch the query
    function fetchArray($SQLQuery) {
        $XSQL = @mysql_fetch_array($SQLQuery);

        if(!$XSQL) {
            return false;
        } else {
            return $XSQL;
        }
    } 

    function numRows($SQLQuery) {
        $XSQL = @mysql_num_rows($SQLQuery);

        if(!$XSQL) {
            return 0;
        } else {
            return $XSQL;
        }
    }

    //function to send a mail on error to the programer
    function ErrorMail($Content) {
        // send an email to the programmer?
        echo '<strong>change the ErrorMail function!!!</strong><br /><br />'.$Content;
    }

    function LogDBError($Statement = "") {
        /*
        // log the database errors?!?
        $qry = 'INSERT INTO dberror
                        SET dberr_number = ''.$this->errNum.'',
                            dberr_text = ''.mysql_escape_string($this->errTxt).'',
                            dberr_statement = ''.mysql_escape_string($Statement).'',
                            dberr_site = ''.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'].'',
                            dberr_ip = ''.$_SERVER['REMOTE_ADDR'].'',
                            dberr_time = ''.time().'',
                            dberr_sent = 0';

        @mysql_db_query($this->DBName, $qry, $this->connection);*/

        echo nl2br('
              dberr_number = ''.$this->errNum.'',
              dberr_text = ''.$this->errTxt.''<br>'
             );

        echo '<strong>change the LogDBError function!!!</strong><br /><br />'.$Statement;
    }

    function ListTables() {
        $result = @mysql_list_tables($this->DBName, $this->connection);

        $table_names = array();

        for($x=0; $x < $this->numRows($result); $x++) {
            $table_names[$x] = @mysql_tablename($result,$x);
        }

        return $table_names;
    } 

    function DBClose() {
        if($this->connection) {
            if(classDataBase::$connections[$this->Idx]['count'] == 1) {
                @mysql_close($this->connection);
            }           

            classDataBase::$connections[$this->Idx]['count']--;
            $this->connection = false;
        }
    }

    function DBCloseAll() {
        if(class_exists('mysql_session_handler')) {
            session_write_close();
        }

        for($i=0; $i<count(classDataBase::$connections); $i++) {
            if(classDataBase::$connections[$i]['connection']) {
                @mysql_close(classDataBase::$connections[$i]['connection']);
            }
        }

        return true;
    }
}

After creating our database class we want to write our own session handling. At first we’ve to create a simple MySQL Table. I called my table cache_session. I want to prefix my example tables with cache_.

CREATE TABLE `cache_sessions` (
  `session_id` varchar(100) NOT NULL default '',
  `session_data` text NOT NULL,
  `session_expires` int(11) NOT NULL default '0',
  PRIMARY KEY  (`session_id`)
) TYPE=innodb;

What is my session handler class doing? It’s a static class to overwrite the session handling. On construct I get the maxfiletime from the php configuration, I register my methods to read, write and destroy the session and create a database connection.

On read / write the session, the class use MySQL to store the information.
mysql_session_handler.php

/******************************************
 * class created by Exi (http://www.technobase.fm/member/1)
 * please find some information about on
 * http://blog.rarecore.eu/nfs-the-unperformed-network-file-system
 ******************************************/
include_once 'classDataBase.php';
if(!class_exists('mysql_session_handler')) {
    class mysql_session_handler {
        static $life_time;
        static $mysql;

        // construct
        function mysql_session_handler() {
            // read the maxlifetime setting from PHP
            mysql_session_handler::$life_time = get_cfg_var('session.gc_maxlifetime');    

            // register this object as the session handler
            session_set_save_handler(
                array(&$this, 'open'),
                array(&$this, 'close'),
                array(&$this, 'read'),
                array(&$this, 'write'),
                array(&$this, 'destroy'),
                array(&$this, 'gc')
            );  

            if(mysql_session_handler::$mysql == null) {
                // create a new database connection or use the already opened
                mysql_session_handler::$mysql = new classDataBase();
                mysql_session_handler::$mysql->Init('db', 'ip', 'user', 'password');
            }
        }

        // open session -> just to have. no action
        function open($save_path, $session_name) {
            global $sess_save_path;
            $sess_save_path = $save_path;
            return true;
        }

        // read session data
        function read($id) {
            $data = '';

            $qry = mysql_session_handler::$mysql->query("select session_data
                                                           from cache_sessions
                                                          where session_id = '".mysql_escape_string($id)."'
                                                            and session_expires >= ".time());

            while($fetch = mysql_session_handler::$mysql->fetchArray($qry)) {
                $data = $fetch[0];
            }

            return $data;
        }

        // insert / update session information
        function write($id, $data) {
            mysql_session_handler::$mysql->query("replace into cache_sessions
                                                              (session_id,
                                                               session_data,
                                                               session_expires)
                                                       values ('".mysql_escape_string($id)."',
                                                               '".mysql_escape_string($data)."',
                                                                ".(time() + mysql_session_handler::$life_time).")");

            return true;
        }

        // destroy a session
        function destroy($id) {
            mysql_session_handler::$mysql->query("delete from cache_sessions where session_id = '".mysql_escape_string($id)."'");
            return true;
        }

        // clean old session
        function gc() {
            // do not clean by an page call, use a cronjob to delete all expired entries every minute!
            /*mysql_session_handler::$mysql->query('delete from cache_sessions where session_expires < '.time());*/
            
            return true;
        }

        // close session
        function close() {
            return true;
        }
    }   

    $SessionHandler = new mysql_session_handler();
}

Now, I want to do some tests…

include 'mysql_session_handler.php';    

session_start();

if(!isset($_SESSION['a'])) {
    $_SESSION['a'] = 1;
    echo 'session "a" not set!';

} elseif($_SESSION['a'] == 10) {
    session_destroy();
    echo 'session destroyed';

} else {
    $_SESSION['a']++;
    echo 'session "a" set!';
}

looks good - works!

The next step is to store the smarty cache in the database. I always write my own classes to overwrite existing open source projects. So I could implement features / settings and software updates doesn’t touch me… I created a classSmarty and opened a database connection using the classDataBase class.

classSmarty.php

/******************************************
 * class created by Exi (http://www.technobase.fm/member/1)
 * please find some information about on
 * http://blog.rarecore.eu/nfs-the-unperformed-file-system.html
 ******************************************/

include_once 'Smarty.class.php';
include_once 'classDataBase.php';
include_once 'mysql_cache_handler.php';

class classSmarty extends Smarty {
    var $mysql;

    function classSmarty() {
        parent::Smarty();	

        $this->template_dir  = '_smarty/templates';
        $this->compile_dir   = '_smarty/templates_c';
        $this->cache_dir     = '_smarty/cache';
        $this->config_dir    = '_smarty/config';

        $this->compile_check = true;
        $this->debugging     = false;
        $this->caching       = true;

        $this->mysql = new classDataBase();
        $this->mysql->Init('db', 'host', 'user', 'pass');
        $this->cache_handler_func = 'mysql_cache_handler';
    }
}

As descripted in the smarty forum I created a function to handle the caching by mysql.

CREATE TABLE `cache_smarty` (
  `smarty_cacheId` varchar(32) NOT NULL,
  `smarty_template` varchar(255) NOT NULL,
  `smarty_group` varchar(255) NOT NULL,
  `smarty_content` mediumtext NOT NULL,
  `smarty_expire` int(11) NOT NULL,
  PRIMARY KEY  (`smarty_cacheId`),
  KEY `smarty_template` (`smarty_template`),
  KEY `smarty_expire` (`smarty_expire`)
) ENGINE=InnoDB ;

mysql_cache_handler.php

/******************************************
 * class created by Exi (http://www.technobase.fm/member/1)
 * please find some information about on
 * http://blog.rarecore.eu/nfs-the-unperformed-file-system.html
 ******************************************/

function mysql_cache_handler($action, &$smarty_obj, &$cache_content, $tpl_file = null, $cache_group = null, $compile_id = null, $exp_time = 0) {
    $db_table = 'cache_smarty';

    // create unique cache id
    $cache_id = md5($tpl_file.$cache_group.$compile_id); 

    switch ($action) {
        case 'read':
            // read cache from database
            $results = $smarty_obj->mysql->query("select smarty_content from $db_table where smarty_cacheId = '$cache_id' and smarty_expire > ".time()); 

            if(!$results) {
                $smarty_obj->_trigger_error_msg('cache_handler: query failed.');
            } 

            $row = $smarty_obj->mysql->fetchArray($results);
            $cache_content = $row[0]; 

            $return = $results;
            break; 

        case 'write':
            // save cache to database
            $results = $smarty_obj->mysql->query("replace into $db_table (
                                                           smarty_cacheId,
                                                           smarty_template,
                                                           smarty_group,
                                                           smarty_content,
                                                           smarty_expire
                                                         ) values (
                                                           '$cache_id',
                                                           '$tpl_file',
                                                           '$cache_group',
                                                           '".addslashes($cache_content)."',
                                                           '".($exp_time == 0 ? time() + 3600 : $exp_time)."' )"); // no expire => expire next hour

            if(!$results) {
                $smarty_obj->_trigger_error_msg('cache_handler: query failed.');
            } 

            $return = $results;
            break; 

        case 'clear':
            // clear cache info
            if(empty($cache_group) && empty($compile_id) && empty($tpl_file)) {
                $results = $smarty_obj->mysql->query("delete from $db_table"); 

            } else {
                /*
                if(!$smarty_obj->mysql->query("delete from $db_table where smarty_expire < ".time())) {
                    $smarty_obj->_trigger_error_msg('cache_handler: clear expired entries query failed.');
                }
                */

                if(strpos($cache_group, '|')) {
                   if(!empty($tpl_file)) {
                      $results = $smarty_obj->mysql->query("delete from $db_table where smarty_template = '$tpl_file' AND smarty_group LIKE '$cache_group%'"); 

                   } else {
                      $results = $smarty_obj->mysql->query("delete from $db_table where smarty_group LIKE '$cache_group%'"); 

                   }
                } else {
                   $results = $smarty_obj->mysql->query("delete from $db_table where smarty_cacheId = '$cache_id'"); 

                }
            } 

            if(!$results) {
                $smarty_obj->_trigger_error_msg('cache_handler: query failed.');
            } 

            $return = $results;
            break; 

        default:
            // error, unknown action
            $smarty_obj->_trigger_error_msg("cache_handler: unknown action "$action"");
            $return = false;
            break;
    } 

    return $return;
}

Now, I want to do some tests again…

include_once 'classSmarty.php';

$smarty = new classSmarty();
$smarty->caching = true;
$smarty->cache_lifetime = 10;

$template = 'smarty.html';
$cache_id = 'smarty|1337';

if(!$smarty->is_cached($template, $cache_id)) {
    $smarty->assign('time', date('H:i:s', time()));
    $smarty->assign('a', rand(1, 9999));
    $smarty->assign('b', rand(1, 9999));
    $smarty->assign('c', rand(1, 9999));
}

$smarty->display($template, $cache_id);

and works again!

And last but not least we want to delete all expired entries. Create a cronjob which is running every minute to do this.

I hope the blog helps you!

Tags: cache_handler_func, hardbase, housetime, network file system, NFS, PHP, session_set_save_handler, Smarty, technobase, trancebase, weareone
Posted in Linux, MySQL, NFS, PHP, Server, Smarty, rarecore | 2 Comments »

The Webradio Experiment

September 25th, 2008

I want to start an experiment and I need your help. My experiment is to push TechnoBase.FM for google ranking. What you’ve to do? It’s very simple. You should link TechnoBase.FM on your on Site, Blog or MySpace Account. Important is that the link name is Webradio.

What’s the background? Normally, a new listener doesn’t know TechnoBase.FM. Our radio leading want to have more and more listener. Often people search for a webradio / internetradio / onlineradio on google and they don’t search on the 20th page. They pick one of the radios from the first or second page. That’s our target. But, we NEED YOU!

The project does not work if you link one of the other radios or the “We aRe oNe” page…

<a href=”http://www.technobase.fm” target=”_blank” title=”TechnoBase.FM – We aRe oNe”>Webradio</a>

I hope that some people support my idea
So Long

Tags: internetradio, onlineradio, rarecore, technobase, we are one, webradio
Posted in rarecore, technobase, we are one | No Comments »

TechnoBase.FM and the story until now

September 18th, 2008

10 month ago, on December 2007, we had some DDOS attacks on our web server. I was on a company Christmas party and could not do anything. The provider turned the server off, my colleagues asked for turning the server on and a couple of minutes later the provider turned the server off again… That was the reason to change our provider.

Well, about 20th December we ordered 2 new servers, 1 database and 1 web server. The 2 servers use an internal 1GBit network and an external network. The serverfarm works, but after some months we had quite an other problem. Too many users on the page at the same time! So, we ordered 2 new servers. 1 web server and 1 application server! And the blog begins

Posted in Linux, Server, rarecore | No Comments »

blog goes online

September 13th, 2008

well,

our new blog is online now. I think it’s nice for everyone to know something about our problems and the following solutions for some webdevelopment issues…

So dudes: Have a lot of fun

Posted in rarecore | No Comments »

NFS, the unperformed Network File System

The Webradio Experiment

TechnoBase.FM and the story until now

blog goes online

Archives

Categories

rarecore

Meta