Dynamic RSS Feed from ColdFusion and bad characters like Â

Are you creating a dynamic RSS Feed in ColdFusion? Maybe you learned some tips from Pete Freitag's blog article. Maybe you have a feed that has worked fine then one day you encounter an issue where bad character(s) are throwing an error in the RSS feed. If you use Internet Explorer, you may see an error like this: An invalid character was found in text content. Line: 71 Character: 271

At the end of this blog I'll describe a simple fix that helped me and hopefully will help you too. Let's say you work at a fictious company and are responsible for publishing news content to the website. Today, each news release is saved as a content record in your CMS database; title, publish date, keywords, content. For the content, each news release contains a text file saved to the web server's file system, and the path to that file is saved in the database. The content file originated in MS Word from your company's communications department. You create a HTML version of the file, mainly using some simple HTML tags; paragraphs, italics, bold, etc. The website's news.cfm page queries the CMS database for a list of news titles, showing the most recent at the top. If a user clicks a news title, they visit the page newsdetail.cfm which does a to display the content file. Maybe this has been working fine for years but now your company wants to offer a RSS Feed of the last 10 news releases. No problem, you read Pete's blog and whip out the page newsrss.cfm coded like this.

<cfset currentoffset = getTimeZoneInfo().utcHourOffset>
<cfquery name="qNews" datasource="CMS" maxrows="10">
select id,title,pubdate,contentpath
from news_table
order by pubdate desc
</cfquery>
<cfsavecontent variable="xmldata"><?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>Company ABC News Releases</title>
<link>http://www.companyabc.com/</link>
<description>Company ABC is a worldwide leader in making news that people like to read.</description>
<lastBuildDate>#DateFormat(Now(),'#ddd, d mmm yyyy')# #TimeFormat(DateAdd('h',variables.currentoffset,Now()),'h:mm:ss')# GMT</lastBuildDate>
<language>en-us</language>
<cfloop query="qNews">
<item>
<title>#title#</title>
<link>http://www.companyabc.com/newsdetail.cfm?id=#id#</link>
<pubDate>#DateFormat(pubdate,'#ddd, d mmm yyyy')# #TimeFormat(DateAdd('h',variables.currentoffset,pubdate),'h:mm:ss')# GMT</pubDate>
<content:encoded><![CDATA[<cfinclude template="#contentpath#">]]></content:encoded>
</item>
</cfloop>
</channel>
</rss>
</cfsavecontent>
<cfcontent type="text/xml" reset="yes" /><cfoutput>#variables.xmldata#</cfoutput>

If your content source is MS Word, it may contain special characters that are not UTF-8 friendly. Examples:

2010–2012 [long/short dash - you are seeing: 2010â€"2012]

€110 bn [Euro - you are seeing: €110 bn]

Castaeda [foreign alpha character - you are seeing: Castañeeda]

value of ? [greek character - you are seeing: value of θ]

[angled quotes - you are seeing: “my quote�]

rental. Minneapolis [double space - you may see nothing obvious or maybe you are seeing: rental. Â Minneapolis]

The fix is very simple, just append charset=utf-8 in this line at bottom of page:

<cfcontent type="text/xml; charset=utf-8" reset="yes" />

For some reason, even though you have the browser does not always interpret it that way. Read this blog to learn more

My page scraper and Twitterfeed

I recently built a small CF app to scrape the Minnesota Twins website. They have a page that is updated once a week (usually Monday) with a new winner of a special prize given to a Season Ticket Holder (STH). I didn't want to remember to check the page every week, so an automated solution was in order.

I created a page scraping template which makes a cfhttp call to the Twins site and parses the HTML for the winner's name and prize. I then check the winner's name against the previous week's winner, and if it differs it saves the new data into the database and emails me the new winner and prize. To make it automated, I created a CF scheduled task that runs once a day and calls the scraping template.

To spread the app to other STH peeps, I expanded it further: 1) Email to subscribers each time a new winner is updated 2) Tweet by @FanInMN each time a new winner is updated 3) RSS feed of the winners

To achieve 2 and 3, the scraping template also updates a RSS feed. This feed is used by Twitterfeed, which automatically tweets the new winner. Thanks to Josh Adam's blog for the tip Twitterfeed, it was very easy to setup.

If you are a Twins STH, you can subscribe here. Go Twins!

Using ColdFusion to Deflate compress a SAML XML authentication message

I am working on a ColdFusion project that uses SAML to authenticate users. I won't get into all the specific application details as they are quite complicated. However, the point of this blog post is to explain the components of the authentication request XML message that gets DEFLATE compressed, Base64 encoded and URL encoded. I hope some of this can help others dealing with SAML and/or Deflate compression in their projects.

To implement "DEFLATE compressed" I knew Java would come into play and I found this resource which explained the Java Deflater class.

This is the example Java code I needed to represent in ColdFusion.

/* Encode a String into bytes */
String inputString = "blahblahblah??";
byte[] input = inputString.getBytes("UTF-8");

/* Compress the bytes */
byte[] output = new byte[100];
Deflater compresser = new Deflater();
compresser.setInput(input);
compresser.finish();
int compressedDataLength = compresser.deflate(output);

I had a hard time finding any help on how I could create the Java byte array "output" variable in ColdFusion. Luckily, I came across this old 2004 blog post from Christian Cantrell. This find was a life saver, thanks Christian! Here is Christian's code:

<cffunction name="getByteArray" access="private" returnType="binary" output="no">
<cfargument name="size" type="numeric" required="true"/>
<cfset var emptyByteArray = createObject("java", "java.io.ByteArrayOutputStream").init().toByteArray()/>
<cfset var byteClass = emptyByteArray.getClass().getComponentType()/>
<cfset var byteArray = createObject("java","java.lang.reflect.Array").newInstance(byteClass, arguments.size)/>
<cfreturn byteArray/>
</cffunction>

Here is my final ColdFusion code that represents the original Java example above.

<!--- setup ColdFusion/Java bytearray variable --->
<cfset emptyByteArray = createObject("java", "java.io.ByteArrayOutputStream").init().toByteArray()/>
<cfset byteClass = emptyByteArray.getClass().getComponentType()/>
<cfset output = createObject("java","java.lang.reflect.Array").newInstance(byteClass, 500)/>

<!--- perform Deflate, Base64 encode, and URL encode --->
<cfscript>
saml_deflate = createObject("java", "java.util.zip.Deflater");
saml_deflate.init(9,true);
saml_deflate.setInput(saml_xml.getBytes("UTF-8"));
saml_deflate.finish();
compressedDataLength = saml_deflate.deflate(output);
data64 = toBase64(output,"UTF-8");
data64url = urlencodedformat(data64);
</cfscript>

Before the ColdFusion code above, you start with the SAML XML message like this, which is saved in the CF variable "saml_xml", and the final result is in the CF variable "data64url".

<cfsavecontent variable="saml_xml"><?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<samlp:AuthnRequest xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
ID="a966242f393474b7d95f4581ff7db131d" Version="2.0"
IssueInstant="2010-05-10T16:24:33.525Z">

<saml:Issuer>http://theissuer.domain.org/app/</saml:Issuer>
</samlp:AuthnRequest></cfsavecontent>

Using ColdFusion with a stubborn MS Access Date/Time field

I have a small application using a MS Access database. Yes, I know all the reasons why it shouldn't be used, let's not go there. Fact is, many developers still use Access for small apps, prototyping, etc, and may come across the same roadblock in which I figured out a solution. For reference, today's date is 3/19/2010 which was used in the query examples shown.

Requirement: Query a table of Jobs (JobTitle, Dept, Salary, etc) that contains a Date/Time field named PostingEndDate. This field is configured as Required = No. That means the admin user who populates records in the table will either supply a date, or may leave the date blank. Think of it as a field that allows NULL in MS SQL. On the end-user side, the query needs to list only current jobs by filtering the records where:
1) PostingEndDate has not passed today's date
OR
2) PostingEndDate is empty (blank value means the Job can be displayed to the user indefinitely)

Here is a cfdump (jobList_raw) of all records:

SELECT JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM tblJob
WHERE     ActiveJob = 1
ORDER BY JobTitle


Problem: no easy way to write the WHERE clause. Here are some attempts:
1) Len(PostingEndDate) = 0
No error, but does not pick up those with blank dates:

SELECT JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM tblJob
WHERE     ActiveJob = 1
AND (Now() <= PostingEndDate OR Len(PostingEndDate) = 0)
ORDER BY JobTitle

2) PostingEndDate = ''
Generates error: [Macromedia][SequeLink JDBC Driver][ODBC Socket][Microsoft][ODBC Microsoft Access Driver] Data type mismatch in criteria expression.
We can't compare the date field to an empty string.

SELECT JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM tblJob
WHERE     ActiveJob = 1
AND (Now() <= PostingEndDate OR PostingEndDate = '')
ORDER BY JobTitle

3) Cstr(PostingEndDate) = ''
Generates error: [Macromedia][SequeLink JDBC Driver][ODBC Socket][Microsoft][ODBC Microsoft Access Driver] Invalid use of Null
This was my attempt to Cast the date to a string using an Access function. Guess not.

SELECT JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM tblJob
WHERE     ActiveJob = 1
AND (Now() <= PostingEndDate OR Cstr(PostingEndDate) = '')
ORDER BY JobTitle

Solution: write a ColdFusion Query of Query (QoQ) to UNION the two conditions into one resultset. Here are the steps that lead to the final solution.
1) I still need a way to get a string representation (varchar) of the date field. So I added another column (PostingEndDate_str) to the Query object.

<cfset QueryAddColumn(jobList_raw,"PostingEndDate_str","varchar",Arraynew(1))>

2) Loop query and populate the new varchar field. This was intended to produce a blank string for what cfdump showed as [empty string]. Then I should be able to use: PostingEndDate = ''

<cfloop query="jobList_raw">
   <cfset QuerySetCell(jobList_raw, "PostingEndDate_str", "#PostingEndDate#",currentrow)>
</cfloop>
NOPE! Still see [empty string] in the new field, THIS BECAME THE HAIR PULLING MOMENT OF THE SOLUTION AT THIS POINT, SO I STARTED GOOGLE SEARCHING.

I decided to try using '-' before and after the value, a trick I noticed in Ben Nadel's blog:

<cfloop query="jobList_raw">
   <cfset QuerySetCell(jobList_raw, "PostingEndDate_str", "-#PostingEndDate#-",currentrow)>
</cfloop>
Now I see "--" for all those [empty string] values, much better. I can work with that in the QoQ.

3) Last is the Query of Query UNION.

SELECT   JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM jobList_raw
WHERE #ParseDateTime(DateFormat(Now(),'mm/dd/yyyy'))# <= PostingEndDate <!--- Lefthand expression will format Now() as: 2010-03-19 00:00:00.0 --->
UNION
SELECT   JobID, DeptID, JobTitle, Salary, PostingEndDate
FROM jobList_raw
WHERE     PostingEndDate_str = '--' <!--- [empty string] dates will contain this value from the QuerySetCell loop executed above --->
ORDER BY JobTitle

Here is the final recordset. It correctly leaves the Architect job filtered out because its PostingEndDate of Mar 9, 2010 has passed. The Mortgage Processor and Supervisor jobs with blank dates are kept in the results!

Excited for special ColdFusion Builder event in St. Paul on March 23

Josh Adams, Adobe Senior Solutions Engineer for ColdFusion, will be presenting live as part of a special CFUG tour. Josh will bring us the latest details and demonstration of the new Eclipse based IDE, ColdFusion Builder. We are planning for a fun event hosted at our usual user group location, Easel Solutions. Adobe is shipping us some special event swag as well, so be there for chance to take home a unique prize. Hope to see a packed house, register now!

www.colderfusion.com/CFBuilder10.cfm

Date: Tuesday, March 23, 2010

Agenda:
5:45 Food and Social
6:30 Presentation begins
8:15 Q & A - Prizes
8:30 After party at local bar TBD

Location: Easel Solutions, St. Paul, MN

Updated 3/24/2010
We had a great event, here are some pics!

ColdFusion Builder - Josh Adams - March 2010

ColdFusion UG Tour coming to St. Paul June 11

I'm thrilled to report that Ben Forta is visiting the Twin Cities again as part of the Adobe worldwide user group tour for the upcoming versions of ColdFusion and Flex. I got the official word a few weeks ago and have been head down in planning mode ever since. Figuring attendance will bust down the doors at Easel Solutions, we needed a bigger venue, so I made a bunch of calls, sent emails, and even visited one potential location. In the end we choose the University of St. Thomas, St. Paul Campus and I think it is going to work out great.

I want to thank the TCCFUG's co-manager, Ben Ellefson, for major work on the event registration website. It's now live and I urge you to get this on your calendar and register now. This is one user group meeting you will not want to miss, plus you'll be fed and could win an IPod touch to boot.

Hope to see you there!

Updated 6/12/2009
We had a great event, here are some pics!

Adobe CF9/Flex4 UG Tour St. Paul 2009

Update your Twitter status from ColdFusion using a JSP tag library

Today I wanted to find an easy way to put a status update into my Twitter account using ColdFusion. I went searching the API docs, and found this page.

http://apiwiki.twitter.com/Libraries

Unfortunately, there are no CF examples listed, boo! However, there are some for Java, so I continued my search in that direction. Eventually I found this JSP taglib, and decided to give it a "Twirl" :)

http://www.servletsuite.com/servlets/twittertag.htm

To use any JSP taglib, you simply drop the .jar file into your /webroot/WEB-INF/lib/ directory. I believe it also requires the Enterprise edition of ColdFusion. A good article that fully explains their usage was done by Charlie Arehart back in May 2002.

After copying the file, you must then restart ColdFusion. This is a must or you will get an error when attempting to import the library.

That's it, I was now ready to test a Twitter post. Here is the sample code I used that will put a new status message in my Twitter account. I saved this in a file twitter.cfm, then browsed to the page on my local machine as http://localhost/twitter.cfm

<cfimport taglib="/WEB-INF/lib/twittertag.jar" prefix="twitter">
<twitter:update user="your_username_here" password="your_password_here" id="result">
My posting to Twitter from CF <cfoutput>#now()#</cfoutput>
</twitter:update>

Upon success, an XML dataset is returned in the "result" variable. If you cfdump it, looks like this.

<?xml version="1.0" encoding="UTF-8"?> <status> <created_at>Wed Feb 04 21:29:26 +0000 2009</created_at> <id>1177644114</id> <text>My posting to Twitter from CF {ts '2009-02-04 15:29:28'}</text> <source>web</source> <truncated>false</truncated> <in_reply_to_status_id></in_reply_to_status_id> <in_reply_to_user_id></in_reply_to_user_id> <favorited>false</favorited> <in_reply_to_screen_name></in_reply_to_screen_name> <user> <id>20062919</id> <name>Twin Cities CFUG</name> <screen_name>TCCFUG</screen_name> <location>St. Paul, MN USA</location> <description>Adobe ColdFusion User Group -of Minneapolis / St. Paul, Minnesota</description> <profile_image_url>http://s3.amazonaws.com/twitter_production/profile_images/75391930/colderfusion_twitter_normal.jpg</profile_image_url> <url>http://groups.adobe.com/groups/bd9082a926/</url> <protected>false</protected> <followers_count>0</followers_count> </user> </status>

If you fail to restart CF, you will see this TagExtraInfo error message:

The TagExtraInfo class com.cj.twitter.strVariable for the update tag could not be found.

The CFML compiler was processing:
* A cfimport tag beginning on line 1, column 2.

This is not the best solution if you are in a shared hosting environment, as they may not install the twittertag.jar file for you. I'm curious what other methods developers have found to accomplish status posts to Twitter.

ColdFusion UDFs long2ip() and ip2long()

Yesterday I was working on porting some PHP code into ColdFusion. The PHP code was using a function called long2ip(), and I researched this PHP Manual website to learn more about it. I needed this function in CF, but couldn't find it at cflib or anywhere else doing a few Google searches. The closest I came was this blog post by Brandon Purcell, which got me started in the right direction. I learned from what Brandon wrote, along with the comments input by Gabriel Malca (if the function doesn't exist) at the PHP Manual site and came up with the logic for my CF version of the function. After I had this working, I figured I better implement the opposite conversion function as well, so I also created ip2long(). Again I found a user comment for when the function doesn't exist to base my function's logic. I submitted these to cflib.org today, so look for them soon.

Examples:
long2ip(3401190660) = 202.186.13.4
ip2long(202.186.13.4) = 3401190660

Here is the code for UDFs below:

<cfscript>
/**
* Generates an (IPv4) Internet Protocol dotted address (aaa.bbb.ccc.ddd) from the proper address representation. Returns 0 if error occurs.
*
* @param longip Numeric value of the address you want to convert. (Required)
* @return Returns a String.
* @author Troy Pullis (tpullis@yahoo.com)
* @version 1, Jan 5, 2009
*/
function long2ip(longip)
{
   var ip = "";
   var i = "";
if (longip < 0 || longip > 4294967295)
      return 0;
for (i=3;i>=0;i--) {
ip = ip & int(longip / 256^i);
longip = longip - int(longip / 256^i) * 256^i;
if (i>0)
         ip = ip & ".";
}
return ip;
}

/**
* Converts a string containing an (IPv4) Internet Protocol dotted address (aaa.bbb.ccc.ddd) into a proper address representation. Returns 0 if error occurs.
*
* @param ip Dotted address value you want to convert. (Required)
* @return Returns a String.
* @author Troy Pullis (tpullis@yahoo.com)
* @version 1, Jan 5, 2009
*/
function ip2long(ip) {
   var iparr = ListToArray(ip,".");
   if (ArrayLen(iparr) != 4)
      return 0;
   else
       return iparr[1]*256^3 + iparr[2]*256^2 + iparr[3]*256 + iparr[4];
}
</cfscript>

CFC for Building a Zip Code Proximity Search with ColdFusion

Back in Oct 2005, SysCon published an article I wrote in CFDJ magazine. Unfortunately, the Webmonkey.com tutorial I originally based my article on has been removed by Wired. I contacted them and hopefully they'll dig it up and repost it under their new wiki site. In the meantime, here is a link to the CFC file zipfinder.cfc used in that article.

One more note... if you make use of my code examples from the CFDJ article, please write your queries using cfqueryparam, which I should have done in the first place.

Twin Cities CFUG November meeting taking shape

We have a great meeting planned for both intermediate as well as brand new CF developers. This meeting is Wednesday, Nov 5th at Easel Training.

Room 1) Kurt Wiersma - BlazeDS
Have you ever wondered what BlazeDS is and what it can do? If so this session is for you. We will cover how you can install BlazeDS inside of CF and how CF can communicate with BlazeDS to make Flex applications get real time messages from ColdFusion.

Room 2) Jason Dean - New Developer Breakout Session
This break out session will be an informal meeting for new developers and for developers who are new to CFML. Bring your questions or just learn the basics of ColdFusion.

Both topics will be presented simultaneously in different rooms. Learn more at colderfusion.com

More Entries

BlogCFC was created by Raymond Camden. This blog is running version 5.9.002. Contact Blog Owner