We’re making the TextYard college bookstore scrapers open source. Any college student with rudimentary coding skills will now be able to take on their local bookstore.
College students are getting ripped off by expensive textbooks, often spending more than $1,000 on them each year. In order to save money, many students are going online to get cheaper prices. However, despite the huge discount from going online, most students still buy their books from the local bookstore.[1]
Why? Because the college bookstore is more convenient.[2] If you want to buy your books online, first you have to search for all of your classes at the local bookstore or its website. Then, you have to find your books one by one at Amazon. College bookstores dominate the textbook market because of their monopoly on the course-to-book data – they are the only ones with a database of which books go with which courses.
A few websites like TextYard have solved this problem for students by scraping this data from college bookstores across the country. Because >99% of college bookstores are using one of 6 online storefront systems, it’s possible to crack the college bookstore code across the United States.
However, it’s not easy. As we detail in the documentation Wiki for our now open source code, the bookstores have been using a bunch of anti-scraping techniques to prevent students from getting unfettered course-to-book data.
For students who are worried about the legal ramifications of scraping the bookstore, we’ve also provided a brief document summarizing why we believe they are on the right side of the law.
It was a pleasure for Rui and I to work on TextYard and help out our fellow students. As we move on to new projects, we hope that we’ve made this little part of the world a little bit better and open.
–Ben Greenberg, Co-Founder of TextYard.com (ben@textyard.com)
PS — I’d be interested in seeing what people do with the course-to-book data. The list of courses by itself has a LOT of potential applications for startups.
Notes
- Green, C. Kenneth. Regulating the bookstore. Retrieved Feb 11, 2012, from http://m.insidehighered.com/views/2009/05/green
- Jonathan Simkin of SwoopThat details the other reasons textbooks are over-priced at http://www.educationnews.org/higher_education/campus_views/157614.html
Dear TextYard,
I go to UT Austin, and textbook prices are insane here, just like at most colleges. I’m trying to build an effective p2p marketplace to enable students to buy / sell books from each other that stay the same semester after semester. Your code should be immensely helpful