After getting poor results from the 'Save Search' feature in Craigslist, I searched for my own data scraper. Target output is an email anytime a new post meets set criteria. 

First, the inevitable 'is this fully legal?' question. I think so! People could get the false idea that scraping is inherently illegal because it is often discussed in the context of scraping confidential info for profit. Here I am only thinking about scraping public data for personal use. 

Linked below is a free tutorial based on hunting apartments in the East Bay. All the variables can be changed to instead look for classic Enduros. I set out to achieve just an email with a link to the new post in as close to real-time as possible. So this solution is a bit of overkill. But it will make it easy to compile some interesting aggregate data like the average asking price for a DT2 or what not. 

Before I start reinventing the wheel does anyone have experience with this or tips to share? I know there are some premium subscription services that offer this but I am only looking for free/open-source tools and ideas.

FB does not allow any scraping at all so this won't work for Marketplace listings. To make things worse they don't have the 'save search alert' feature because they love buyers sitting in front of the screen hitting refresh all day. If anybody has a workaround for that, it is the Holy Grail in the lazy man's quest for barn-find Enduros. I am happy to share everything I find and create with anyone not trying to buy a 60's or early 70's Yamaha or Ducati 250 single in Maine or New Hampshire.  

Happy Hunting and Happy Holidays to all!

towardsdatascience.com/web-scraping-crai...utorial-c41cea4f4981


-Bubs