Posts Tagged ‘Yahoo! Pipes’

Weekly Link Roundup – 2

Links published on May 04, 2009:

Math world, simple mental calculations or what’s going on with education?

Most interesting math blog post I have read for a long time. Have a read guys and tell me how you find it.

Programmer Competency Matrix:

http://www.indiangeek.net/programmer-competency-matrix/

Panoramic Image & Viewer related resources:

http://krolik.net/PIV/
http://flashpanoramas.com/player/
http://www.mojavelinux.com/cooker/demos/panojs/
http://www.eurofresh.se/test/shockwpanos.htm
http://www.michielpost.nl/panorama/

File conversion utility for panoramic photos:

http://www.outerspace-software.com/bixorama.html

Automatic image stitching:

http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html

Panoramic Image App (Commercial):

http://www.autopano.net/

Panorama Gallery:

http://www.cs.ubc.ca/~mbrown/panogallery/panogallery.html

Panoramic Viewer:

http://s3.envato.com/files/4810/index.html

TDD Links:

http://codebetter.com/blogs/darrell.norton/articles/50337.aspx
A Conversation with Martin Fowler
Mocks Aren’t Stubs
Learn about .NET Unit Testing with Typemock

Old Version Downloads:

http://www.oldapps.com/

10 Youtube URL Tricks You Should Know About

http://www.makeuseof.com/tag/10-youtube-url-tricks-you-should-know-about/

100 Best JavaScript Resources

http://www.spoonfeddesign.com/100-best-javascript-resources

Performance killer: Disk I/O

http://nathanaeljones.com/11061_Performance_killer_Disk_I_O

Top 20 Best Agile Development Books, Ever

http://www.noop.nl/2008/06/top-20-best-agile-development-books-ever.html

Top 100 Best Software Engineering Books, Ever

http://www.noop.nl/2008/06/top-100-best-software-engineering-books-ever.html

Top 20 Programming Books

http://yaneshtyagi.blogspot.com/2008/04/top-20-programming-books.html

Top 10 Concepts That Every Software Engineer Should Know

http://blog.csdn.net/andylin02/archive/2008/07/26/2716304.aspx

Securing ASP.NET Applications

http://www.novologies.com/post/2009/04/08/Securing-ASPNET-Applications.aspx

The ADO.NET Entity Framework must read:

Introducing the Entity Framework [MSDN]
The ADO.NET Entity Framework Overview
ADO.NET Tech Preview: Entity Data Model

Introduction to the Entity Framework [Code Project]

Windows Communication Foundation must read:

Fundamental Windows Communication Foundation Concepts
What Is Windows Communication Foundation?

A Guide to Designing and Building RESTful Web Services with WCF 3.5

Howto Dynamically Insert Javascript And CSS:

This is a short and sweet little tutorial to show you how to dynamically insert a new Javascript (or style sheet) into your web pages, and explores how you can exploit cross-domain includes in your applications.

As JSON and DHTML start to get pushed more and more to the forefront of web-based applications, the web designer is faced with a new problem: how to dynamically insert a script element into an existing web page. It won’t take long to figure out that ajax loads and innerHTML injections won’t work.

http://www.hunlock.com/blogs/Howto_Dynamically_Insert_Javascript_And_CSS

Yahoo! Pipes and JQuery : Goodbye Same Origin Policy

http://www.badlydrawntoy.com/2008/07/08/yahoo-pipes-and-jquery-same-origin-policy/

URL Shorteners:

http://is.gd/
http://tr.im/
http://ow.ly/url/shorten-url
http://bit.ly
http://tinyurl.com

Interview Tips:

http://interviewtips.org/

LINQ to Entities:

http://www.linqdev.com/PublicPortal/publicportal/linq-to-entities.aspx

LINQ Learning Guide: LINQ to Entities

http://searchwindevelopment.techtarget.com/generic/0,295582,sid8_gci1299600,00.html

The ADO.NET Entity Framework Overview:

http://msdn.microsoft.com/en-us/library/aa697427(VS.80).aspx

Parallelism Videos Galore

http://blogs.msdn.com/pfxteam/

Writing LINQ Queries with the Entity Framework:

http://www.informit.com/articles/article.aspx?p=1237071

ASP.NET Dynamic Data Overview

http://msdn.microsoft.com/en-us/library/cc488546.aspx

Inheritance and Associations with Entity Framework Part 1

http://mosesofegypt.net/post/Inheritance-and-Associations-with-Entity-Framework-Part-1.aspx

Model Domain Objects with the Entity Framework

http://visualstudiomagazine.com/features/article.aspx?editorialsid=2425

Lambda the Ultimate : Programming Languages Weblog

http://lambda-the-ultimate.org/

Introduction to Domain Specific Languages

http://www.infoq.com/presentations/domain-specific-languages

Good blog site on Patterns:

http://blogs.msdn.com/erwinvandervalk/default.aspx

Pattern site on Hexagonal architecture

http://alistair.cockburn.us/Hexagonal+architecture

JSINQ – LINQ to Objects for JavaScript

http://www.codeplex.com/jsinq

JSINQ: LINQ style access for JavaScript objects

http://ajaxian.com/archives/jsinq-linq-for-javascript

LINQ to SQL: SQL-CLR Type Mismatches (LINQ to SQL)

http://msdn.microsoft.com/en-us/library/bb386909.aspx

LINQ to SQL : Object Identity (LINQ to SQL)

http://msdn.microsoft.com/en-us/library/bb399376.aspx

Most interesting math blog post I have read for a long time. Have a read guys and tell me how you find it.

Scraping your way to RSS feeds!

I was looking for a way to get regular updates from a job site about a particular category even though the site doesn’t offer any sort of feed.

Then I stumbled upon a site called Feedyes.com.

What I basically did was to have an RSS feed ready for the site. It’s pretty elementary with the help of the above site really. You don’t even need to register in order to create an RSS feed for a certain site.

Only problem was that I didn’t have the RSS feed in XML format. I had to go to the web site to view so. Also the feed couldn’t really be customized in any ways.

There’s another site named Page2rss.com which does pretty much the same. Mind you none of the above sites are perfect yet they do a reasonable job of it.

So I googled a bit more and stumbled upon Feed43.com which let me actually write expression for creating the feed.

Here‘s what I came up with as an RSS feed version of this page. It lets you use ‘search patterns’ using regular expression and ‘output templates’. It’s a handy site even with all its limitations for unpaid package like polling intervals, maximum feed limit etc. Do give it a try.

I know there are several good articles like Creating a generic Site-To-RSS tool, When RSS Fails: Web Scraping with HTTP and How To: Scrape a Web Page to RSS Feed for doing the kind of the same.

What’s more I don’t know if you know this but both Yahoo! and MSN provides search result in RSS format.

Here’s the result using Yahoo! web search service for ASP.NET MVC and here‘s MSN’s version for the same.

But of course, it would help if Google was to have an XML feed of their normal Search engine positioning (SERPS) like Yahoo! & MSN do.

What it does provide though is an RSS feed for searching blogs. Try this.

There’s another gem I figured which actually lets you run XPath query for scraping into a web page for RSS. It can be used to search in an HTML document in a pretty straightforward way.

Well this has been a very long ride for scraping your way to another site but what if you want to stop others doing the same :). Enough of RSS Scraping, Scavenging, Stealing, and Content Theft, no? Talk about having a dose of one’s own medicine, right?

Anyway, have a look at What Do You Do When Someone Steals Your Content or better still have a read about the antonym of Scraping in IT terminology Information Trapping.

To wrap things up, do remember there are words like Copyright and Intellectual property / Intellectual Property Protection in the dictionary :). So use it in a positive way and enjoy the Scrapventure!

Update on 9th April, 2009: It was unfair on my part to leave off tools like Yahoo! Pipes and Feedity.com. While Yahoo! Pipes is a less than straightforward means to achieving our objective, it has powerful features like Visual query development which are missing from the rest. But I think what makes Yahoo! Pipes unique is that you can chain together arbitrary number of previous queries (pipes) and thus mash them up into one which would have all your filters/queries. It also provides input facilities. More on Yahoo! Pipes later on subsequent post perhaps when I would guide you through the process. Feedity.com, on the other hand, is a very straightforward means to achieving what we want. It’s quite efficient and intelligent with parsing too. Give it a try.

Update on 16th April, 2009: Microsoft Popfly mashup creator is another candidate for honorable mention 🙂