The problem turns out to be simply that the symfony 1.2 swToolkit plugin uses Zend Mail instead of phpMailer and Zend Mail spits out encoded text when you ask it for the email text or HTML contents.
So the simple fix is to wrap the data you get out of Zend Mail with the obscure quoted_printable_decode() function as shown here:
$message = quoted_printable_decode($zendEmail->getBodyHtml()->getContent());
$text = quoted_printable_decode($zendEmail->getBodyText()->getContent());
There, that gets rid of all those =0D=0A characters everywhere!
Note that you would think you could avoid this simply by passing the line ending explicitly, but it does not work:
//this does *NOT* work!
$message = $zendEmail->getBodyHtml()->getContent("\r\n");
$text = $zendEmail->getBodyText()->getContent("\r\n");
[ add comment ] | [ 0 trackbacks ] | permalink | related link |
( 2.5 / 4 )I have battled postfix, qmail and sendmail for years - and my latest trouble involved sendmail trying to route emails to local (non-existant) users, even though the domains in question were not in the local domains config.
After running around in config circles for a couple of hours it turns out that if you set the server's hosts to include your domains, sendmail will assume local delivery for those domains.
That is even if you have sendmail configured not to (ie: in /etc/mail/local-host-names). It even ignores your virtusertable with explicit instructions to forward mail for those domains elsewhere.
Go figure. Hope this helps someone.
[ 1 comment ] ( 4 views ) | [ 0 trackbacks ] | permalink | related link |
( 2.8 / 8 )That's one of the problems with 'enterprise' websites - huge databases. Knowing that everything is backed up, it is occasionally tempting to run some ad-hoc handmade 'looks about right' SQL statements every now and then - and occasionally they don't work out too well.
As it happens, restoring a table from a 5GB SQL backup can actually be a pretty time consuming task, not to mention tedious.
Faced with this slightly daunting task I did what I usually do when faced with daunting tasks - checked how other people did it :)
So a big thanks to ThatsLinux for sharing a nice little shell script that takes the pain out of extracting one little table from a mammoth sql script. Kudos.
ThatsLinux
[ add comment ] ( 12 views ) | [ 0 trackbacks ] | permalink | related link |
( 2.9 / 37 )All right now Google are installing Chrome inside Internet Explorer so now Microsoft can claim to have a standards compliant browser. But what I'm excited about is viewing it all inside Firefox with the ieTab plugin!
That's Firefox rendering Internet Explorer rendering Chrome.
Now somebody create a chrome plugin that uses the Opera rendering engine or - even better - the Firefox rendering engine!
Whoop!
[ 2 comments ] ( 685 views ) | [ 0 trackbacks ] | permalink | related link |
( 2.8 / 63 )When I get the urge to run a symfony command from an explorer window I have a few options...
Command prompt here
Well this is handy but you still have to type out your command. Fortunately I have PowerCmd which lets you save common commands and make them toolbar buttons.
Directory Opus
My favourite explorer replacement lets me type commands directly into the toolbar - and it *remembers* the history of commands. Nice.
XYplorer
Another good file manager - and a bit more lightweight than Directory Opus - has a great scripting engine. Check this out:
::if (confirm("propel-build-ALL?!")){
run 'php symfony propel-build-all',<curpath>
}That's not bad. And you can make it a favourite (category) that will execute on the current activated folder. Again, nice.
PhpEd
Well this isn't exactly an explorer replacement, but seeing as though I never close my IDE it is technically a file manager ;) And as I posted here, you can add scripts to the right-click menu. The biggest drawback is that you can't put < characters in your scripts, which means you can't load sql files into mysql without requiring a seperate DOS batch file.
[ add comment ] ( 9 views ) | [ 0 trackbacks ] | permalink | related link |
( 3 / 59 )This is something that has been discussed in the Symfony google groups but was unresolved - how to modify a view configuration based on the environment. Some developers say you should not be doing this, as the view should be the same in all environments, but let me explain why I needed do do it...
I have some javascripts loaded from google to speed up my sites - anyone who has used the Yahoo YSlow plugin for FireBug will be familiar with the "F" rating for content delivery networks (CDN) score. But I have a developer on my team who works offline and wants to be able to develop locally without having to constantly modify all the view.yml files (and there are a few!) for each app.
So, although it would be nice to be able to specify an environment in the view.yml the same way you can in the settings.yml, some folks forget that you can put PHP code in your view.yml to be evaluated at runtime (or at least read from the cache at runtime!).
So, this is how we manage to serve google's minified high speed Javascript libraries in the "live" environment but use local copies elsewhere:
default:
javascripts:
- <?php echo (strpos(SF_ENVIRONMENT,"live")!==false?'http://ajax.googleapis.com/ajax/libs/prototype/1.6.0.3/prototype.js':'/js/prototype').PHP_EOL ?>
- <?php echo (strpos(SF_ENVIRONMENT,"live")!==false?'http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.2/scriptaculous.js?load=effects':'/js/scriptaculous/scriptaculous.js?load=effects').PHP_EOL ?>
Damn, that was easy, wasn't it?
:-)
[ add comment ] ( 4 views ) | [ 0 trackbacks ] | permalink | related link |
( 2.9 / 78 )Well, in case you were wondering, I managed to get around my scalability problem by focussing on PHP's strengths and side-stepping it's weaknesses. So in this particular problem I was facing 2 primary bottlenecks - yaml parsing and propel memory leaks.
I got around the yaml issue by using some custom string hacking - and PHP's string functions are pretty quick. By using split() and strpos() I could slice up my big yaml file into more digestible sizes (see previous post).
The propel issue, however, was always going to be more problematic. I considered removing all my ORM calls and either using delayed inserts or LOAD DATA INFILE statements. Both of which I have used successfully in the past, but implementing it in this case meant a complete rewrite of over 2000 lines of tested code.
So, I decided to use a similar approach to the ORM. Like YAML, Propel 1.2 is very good at what it does inside small to medium processes. It just isn't very efficient when used in big jobs - like inside large loops. So instead of trying to process my massive array that I managed to construct from the original yaml, I sliced it up and sent smaller yaml files back into the queue.
What this means is that instead of having a single item of 13,000 recipients sitting in the queue I have 130 items of 100 instead. My queue processing task can now batch 100 emails at a time and then terminate, thereby freeing up memory for the next batch. While there is some extra overhead in writing files to disk and duplicating some of the yaml, the benefits are well worth it. The end result is a scalable, fast and robust system that can handle almost any request by simply slicing it up into manageable chunks. TIme and money saved, client happy :)
[ 4 comments ] ( 1536 views ) | [ 0 trackbacks ] | permalink | related link |
( 3.1 / 89 )This is a problem I often come up against - sending email to thousands of recipients. The way I usually handle it is to do as little as possible in the master loop.
The best technique I have found is to defer database writes by writing to a temporary textfile in memory (using php://memory) and then using LOAD DATA INFILE to write contents to the database. That puts all the email data into a database queue, allowing a cron task to send the emails - say 50 per minute.
Using this technique I can send 50,000+ emails quite easily. The queue function takes under a minute to run and the cron job hacks away at the queue for about 24 hours or so.
The problem I have with this technique is that it evolved via a process of whittling away at the various framework layers used in my "enterprise applications". Framework layers supposedly designed to provide scalability.
Now you can probably tell from this blog that I use Symfony extensively (although I have dabbled with other frameworks, I always come back to Symfony). Perhaps it is due to the fact that my largest projects (over 1,000,000 lines of code) are locked into Symfony 1.0 and - more importantly - Propel 1.2. Arguably, what I should be doing is writing a Symfony plugin that performs large data processing using the techniques I've described, but it seems a shame to have a detailed, documented and unit-tested ORM that can only be used for serving small jobs like "web pages", and must be bypassed for any data intensive stuff. Isn't that a core requirement of "enterprise level" scalability?
Anyway, I'll try to end the rant with some code. Last night I wrote a custom yaml parser because the sfYaml::load() method in Symfony 1.0 (which uses spyc) was taking over 20 minutes to load a 1MB yaml file. I managed to load it in about 5 seconds. Now, I'll admit it's not really a yaml parser but more of a key-pair extractor but because it doesn't use any regex it is waay faster than the old sfYaml.
(NB: I couldn't install syck on my CentOS4 server and the newer sfYaml component from 1.2 failed to load yaml collections in the format shown below).
The yaml (multiply this by 10,000):
Recipients:
- title: 'Mr'
firstname: 'Bob'
lastname: 'Dobbs'
company: 'Subgenius Network'
email: 'bob@subgenius.com'
- title: 'Mrs'
firstname: 'Jane'
lastname: 'Dobbs'
company: 'Subgenius Network'
email: 'jane@subgenius.com'
The recipient extractor method:
public static function extractRecipients($yaml)
{
//separate recipients from yaml
$recipients_string = "Recipients:";
$recipients_start = strpos($yaml,$recipients_string);
$recipients_end = strpos($yaml,"bcc:");
$recipients_yaml = substr($yaml,$recipients_start,
($recipients_end-$recipients_start));
$yaml_start = substr($yaml,0,$recipients_start);
$yaml_end = substr($yaml,$recipients_end);
$yaml = $yaml_start . $yaml_end;
//load yaml
$yaml_array = sfYaml::load($yaml);
//get recipients array
$recipients_yaml = str_replace($recipients_string,"",
$recipients_yaml);
$recipients_array = split("-",$recipients_yaml);
//loop over recipients to get getails as array
$recipients_list = array();
foreach ($recipients_array as $r1) {
//loop over each line item
$r1_array = split("\r\n",$r1);
$my_recipient = array();
foreach ($r1_array as $r2) {
if (strpos($r2,":")!==false){
//extract key pair from line item
$r2_array = split(":",$r2);
if (sizeof($r2_array) == 2){
$key = str_replace("''","'",
trim(trim($r2_array[0]),"'"));
$value = str_replace("''","'",
trim(trim($r2_array[1]),"'"));
$my_recipient[$key] = $value;
}
}
}
if (sizeof($my_recipient) > 1){
//add this recipient to list
$recipients_list[] = $my_recipient;
}
}
//add recipients to yaml data
$yaml_array[trim($recipients_string,":")] = $recipients_list;
//return yaml data
return $yaml_array;
}
[ 4 comments ] ( 2097 views ) | [ 0 trackbacks ] | permalink | related link |
( 3 / 91 )I have enabled comments on the blog after a few requests for it... play nice!
[ add comment ] ( 7 views ) | [ 0 trackbacks ] | permalink | related link |
( 3 / 93 )I recently had to integrate a forum into a Symfony 1.0 app and, having selected PhpBB3 for the job, went about writing an authentication module. Here's how...
1. Create the auth module
Lets say I want to call the module 'symfony'. I create a file called auth_symfony.php and drop it in the phpbb /includes/auth folder.
2. Write the autologin method
Because I want my forums to be on the same domain, I have saved the phpbb files in web/forum. This way I have access to the session variables stored in my symfony app. So when a user goes into the forums, the autologin method can be used to interrogate my smyfony session.
In a nutshell, the autologin method needs to verify the user's session, grab their details and log them into phpbb. So as to avoid having to manage forum users from my symfony app, I also create new phpbb users here if one doesn't already exist for the current user.
Note that phpbb needs its own session, so we need to switch between sessions here as well.
/**
* Autologin function
*/
function autologin_symfony()
{
include_once('includes/functions_user.php');
global $db, $config, $user;
$sess = session_name();
session_name('symfony');
session_start();
$sfSession = $_SESSION['symfony/user/sfUser/attributes']['userData'];
@session_name($sess);
@session_start();
if (isset($_REQUEST['admin'])){
$_SESSION['admin'] = $_REQUEST['admin'];
}
if (isset($_SESSION['data'])){
return $_SESSION['data'];
}elseif (!isset($sfSession['username']) &&
!isset($_SESSION['admin'])){
header("Location: /");
exit;
}elseif (isset($sfSession['username'])){
$user_row = array(
'username' => $sfSession['username'],
'user_password' => phpbb_hash($sfSession['username']),
'user_email' => $sfSession['email'],
'user_type' => USER_NORMAL,
'group_id' => 2
);
$sql ='SELECT *
FROM ' . USERS_TABLE . "
WHERE user_email = '"
. $db->sql_escape(utf8_clean_string($sfSession['email']))
."'";
$result = $db->sql_query($sql);
$row = $db->sql_fetchrow($result);
$db->sql_freeresult($result);
if ($row){
// Successful login...
$data = array_merge($row,array(
'status' => LOGIN_SUCCESS,
'error_msg' => false,
'autologin' => 1,
'user_row' => $row
));
$_SESSION['data'] = $data;
return $data;
}else{
//check for existing name
$sql ='SELECT *
FROM ' . USERS_TABLE . "
WHERE username_clean = '"
. $db->sql_escape(utf8_clean_string($sfSession['username']))
."'";
$result = $db->sql_query($sql);
$row = $db->sql_fetchrow($result);
$db->sql_freeresult($result);
if ($row){
//randomise username if duplicate found
$user_row['username'] = $user_row['username']
. " - " . rand(1000,9999);
}
//create new user
$user_id = user_add($user_row);
$user_row['user_id'] = $user_id;
$data = array_merge($user_row,array(
'status' => LOGIN_SUCCESS_CREATE_PROFILE,
'error_msg' => false,
'autologin' => 1,
'user_row' => $user_row,
'user_type' => USER_NORMAL,
'group_id' => 2
));
$_SESSION['data'] = $data;
return $data;
}
}
}
Note also that the $_SESSION['data'] array is used to avoid hitting the database everytime the user loads a new page in phpbb.
3. Create admin login
You might have noticed that the above code looks for a session var called 'admin'. I use this to track whether a user has come from the backend. This is used to display the phpbb login form. Otherwise the user is simply sent back to the symfony app login.
/**
* Login function
*/
function login_symfony($username, $password)
{
global $db, $config, $user;
$sql ='SELECT * FROM ' . USERS_TABLE
. " WHERE username_clean = '"
.$db->sql_escape(utf8_clean_string($username))
."' and user_password = '".md5($password)."'";
$result = $db->sql_query($sql);
$row = $db->sql_fetchrow($result);
$db->sql_freeresult($result);
if ($row){
$data = array_merge($row,array(
'status' => LOGIN_SUCCESS,
'error_msg' => false,
'user_row' => $row
));
$_SESSION['data'] = $data;
return $data;
}else{
return array(
'status' => LOGIN_ERROR_PASSWORD,
'error_msg' => 'LOGIN_ERROR_PASSWORD',
'user_row' => array('user_id' => ANONYMOUS),
);
}
}
4. Create logout method
This simply wipes both sessions and redirects the user to the symfony app homepage.
/**
* Logout function
*/
function logout_symfony($user_row)
{
$sess = session_name();
session_name('symfony');
session_start();
session_destroy();
@session_name($sess);
@session_start();
@session_destroy();
header("Location: /");
exit;
}
5. Create validate session method
Last but not least this method is used to check that the current user session is valid. Not 100% required, but a good precaution.
/**
* Validate session function
*/
function validate_session_symfony()
{
$sess = session_name();
session_name('symfony');
session_start();
$auth = $_SESSION['symfony/user/sfUser/authenticated'];
@session_start($sess);
@session_start();
//always redirect to home
$admin = isset($_SESSION['admin']);
if ($admin || $auth){
return true;
}
return false;
}
6. Configure
You'll need to log into the phpbb ACP and set the authentication module to your custom module. Then you need to bypass symfony in the .htaccess for the /forum URL
RewriteRule ^forum/.*$ - [PT]
RewriteCond %{REQUEST_URI} !^/forum
And that's basically it :)
[ add comment ] ( 46 views ) | [ 0 trackbacks ] | permalink | related link |
( 3 / 100 )Next
Avatar
