We are currently setting up our Support web site which is available at http://support.agentstorm.com/. We decided that this will replace both the Wiki and Community Forums. The support site will give us greater control over your support tickets, provide a searchable frequently asked question database as well as a public forum for any issues our users may have.
Thanks to #tenderapp for providing this excellent solution!
Last week we moved all our infrastructure to Rackspace and now that we are nearing the end of the migration I wanted to let everyone know about some of the architectural changes we have made to make Agent Storm faster, and more scalable for the future.
The biggest change is probably the introduction of two load balancers, lb-01 and lb-02 and run Nginx. There is only ever one of these load balancers active while the other waits to take over should the active one fail in any way. These load balancers perform two crucial jobs, they spread the load of the inbound web traffic across the application servers, and also offload the SSL processing from the application servers.
The application servers are in them selves simple web servers that process requests received via the load balancers connecting to the MySQL server as needed. Currently we are running two application servers for redundancy. For certain operations that require a long running or background operation such as RETS Imports, Address Geocoding etc, the application servers place that command in a queue on the Redis server which is appropriately named queue-01.
All our background workers are written in Python is (in our minds at least) in its essence with background workers. It has open source libraries for pretty much anything you can imagine, its fast, requires a minimal memory footprint and has great multithreading support. Our background workers currently run on servers named queue-01 and queue-02. The workers are all managed and scheduled using pyres which is a Resque clone written in Python.
One major part of the import system we store in Redis is a Set containing a list of files we have stored on our CDN. Storing this list in Redis gives us quick and easy lookups to see if a file is already stored on the CDN before we go and upload it again, wasting both time and bandwidth. This list can become out of sync so periodically we ditch the list and reload it from the CDN. At first you might think that this has some serious scalability issues but we only store each image once no matter how many times that photo is assigned to properties that have been imported by various RETS imports. At most we expect to store about 10-20 million records which by current estimates will use between 500Mb to 1Gb of RAM which is not that much given the speed improvement gained in the import workers and functionality it provides. (i.e During an import we may perform over 100,000 internal Redis queries on images alone)
Conclusion
What we have now should provide us with a scalable infrastructure that allows us to scale up as demand increases and also provide a great base to build new features on. We are using truly modern programming practices and emerging technologies to build a platform which is ahead of our competition. Our average local response time is 0.13 seconds and average total response time from connection open to connection close of 0.5 seconds.
Recently we have been adding a lot of extra details you can add to your contacts, such as Instant messaging addresses, Web sites and Social Network sites and as a result the contacts page was starting to look very cluttered. We decided to combine the “Add XYZ” buttons in to one.
The result is a split button, Edit Contact which brings up the edit contact screen as before and a drop down list giving you access to add additional information to your contact such as Postal Addresses, Phone Numbers, Email Addresses, Instant Messaging Addresses, Social Networks and URL’s.
Below you can see the difference with before and after screenshots.
Before
After
Like it? Don’t Like it? Let us know in the comments.
Today we are announcing you can now import your contacts in to Agent Storm from both Plaxo and Google. On the contacts screen you will see a new icon “Import Contacts” in the top right allowing you to choose your address book provider and import your contacts. All authentication is handled using oAuth so we will never ask for your password to any third party service.
We were also hoping to release the importer for Yahoo contacts at the same time however due to some serious locking down of oAuth on Yahoo’s servers the yahoo importer will be delayed for a while.
We have more importers that will be released very soon including Hotmail, Yahoo, AOL, Highrise, Apple Mail Outlook and Mozilla Thunderbird.
If your an application developer, remember you can access Agent Storm data using oAuth, allowing you to build applications that use information stored in a users Agent Storm account.
Firefox 4 Beta 1 was released this week and it is surprisingly faster, whether thats because all my extensions are now disabled due to not being compatible with this version or if its under the hood time will tell. This beta is more interesting to windows users due to tabs on top feature but unfortunately for us Mac users the theme has not been updated as of yet to prettify the tabs on top feature.
I did however find a mock up of what this feature will look like on the Mozilla Wiki. Is it enough to go back to Firefox from Google Chrome? Have to wait for the next Beta and see.
We noticed that over the last week the Wiki had been hit by a few thousand spam bots. Although we thought we was watching any and all page modifications we apparently where not. We have since installed captchas for all unregistered user page edits and also on the registration form itself. We expect this to stop the spam bots in their tracks but we will keep monitoring it.
We are also going on a major documentation update this week so expect a lot of fresh content on the Wiki!
If your a Real Estate Developer, not the type that builds houses but one that has some great ideas on how to display MLS data for the web then Agent Storm can help you out. We all know how much of a pain getting access to IDX data can be, let alone developing products and solutions around them.
We cant help you with getting access to the data but we can help you make use it. Today Agent Storm announces support for oAuth on its API, we know we are one of the first Real Estate services to implement oAuth for providing secure API access to 3rd party developers.
What is oAuth?
OAuth is an open protocol that allows users to share their private resources (e.g. photos, videos, contact lists) stored on one site with another site without having to hand out their username and password.
What does this mean for developers?
Developers can now build and market their own add-on solutions to the Agent Storm platform. If you have a cool idea how to display property data, or a ingenious idea for map search then now its possible using the Agent Storm API’s.
The Agent Storm API’s currently allow:
How do I get started?
First you will need an Agent Storm account, once logged in you will see a “Developer” link in the top right hand corner. Clicking this link will take you to a page where you can request an oAuth Consumer Key and Secret, these keys are valid for requesting access to any agent storm application domain and not just the domain it was requested from. See the oAuth spec for guidance, you will need to talk to the following oAuth endpoints :
* Replace “acme” with the users agent storm sub domain.
Where can I get Help?
We have a new Google Group setup for helping Developers build solutions on the Agent Storm platform available here. We will also be updating our Wiki with more information over the coming weeks.
oAuth support is still in beta, where as it should work for the majority of implementations it may not work for everyone. If you have a problem using oAuth with Agent Storm just get in touch wither with the Developer Mailing list, or by emailing us as support@agentstorm.com.
We enjoy keeping everyone updated with what is coming soon and what you can look forward to as we progress on Agent Storm. A few weeks ago we blogged about Android Apps for Real Estate Agents and REALTORS but the pickings where very slim indeed. But fear not Agent Storm will soon be available on your Android phone using our Native Agent Storm app for Android.
We have a long way to go yet but some of the features you can expect are:
We expect our Android App to be released in the coming week or so so stay tuned!
If your looking for a new web site for your Real Estate business then look no further. Agent Storm is offering 3 of our lucky blog readers a FREE professional custom designed web site and technology overhaul using the Agent Storm system. Your web site will be professionally designed, Search Engine Optimized and showcased on the Agent Storm web site to demonstrate our products. You will retain complete ownership of the leads generated by the web site and we will also give you 2 years free use of the Agent Storm System with unlimited contacts and properties from up to 3 different MLS systems.
Obviously there is a catch, in return you must agree not to use or promote a competing product on the site that was designed by us for you.
To be considered get in touch now via our Contact Form and let us know your current web site address and we will be in touch with you to consider you and your web site for a web design overhaul.
After reviewing how the site is being used we decided that the width was an issue. Given that the majority of monitors these days are wide screen it makes sense to use a fluid width (100% of the screen width) rather than the fixed width we where using before. Now we are using more screen real estate allowing you to use the full potential of your monitor. More useable space = more productivity.
Below is a screen shot of what we will be rolling out later tonight. As you can see we have moved the toolbar and customizable logo to the left hand side of the screen and made the toolbar vertical. Going forward this enables a lot more space for us to use for menus for the features we will be adding to the system in the coming weeks. We hope this design change enables better productivity and allows you to use Agent Storm faster.
We would love to hear any feedback you have on this design change either in the comments or via our UserVoice Forum.
EDIT: The changes mentioned above have now been rolled out.
This is the Blog from Agent Storm a Real Estate Marketing system designed for REALTORS® and Real Estate Agents.
Here on our blog we discuss various aspects of Real Estate Technology and how it affects you, the Realtor, with tips on how to take advantage of it.