November 26, 2014

How Web Services world affect LAMP Stack

In connection to my previous post I was thinking how Web 2.0 with its massive use of Web Services affect LAMP Stack ? Well actually there are serious difference both for client and server application development which you might want to take into account.

If you’re developing Web Service Server application you will most likely be dealing with two types of requests

  • Real time requests. These are executed during web page generation on the client and add up to its response time. These are important to be answered as fast as possible. If you can’t service them it time it might be better to return partial results or error code – not everyone is good at handling timeouts properly. One more nice feature you might consider is allowing to execute multiple operations with single request. This both allow you to service them in parallel and reduce number of round trips which are very expensive.
  • Batch Operations. Web services are also often used in batch mode – where client just needs to process certain data using Web Service. In this case you do not really care about response time of individual requests it is throughtput which is important. Such requests are also might be treated with idle priority – batch processing often can be done during the night when real time load is light. Actually even certain web requests may fall into this category – for example Search Engine bots often much more patient than users and can wait for page generation longer. Of course Batch operations would also benefit from ability to submit multiple requests at the same time, as well as keep alive.

There are not many web services out where which make this difference but I think it would be quite useful especially with Business to Business web services which might expect competent users using them.

There are also number of changes on the client application, which is frequently yet another web application:

  • More workers required. In my post yesterday I mentioned you might need only few “workers” – actively working web server processes, if you do not have to wait for slow clients. Now you however also have to wait for remote services during web page generation which must be done while worker is active. So you end up needing much more workers, especially dealing with slow web services.
  • Timeouts. Web services might get slower themselves or network connectivity to them might slow things down so proper timeout handling is a must. If you use multiple web services you might want to use timeout for total call times during same web page generation. For example you might want your page to be generated in 5 seconds and if first web service responded in 4 seconds second one will have only one second to respond. You might use different values but it is better to display page with partial information than no page at all. You also might use certain flags to avoid querying service if it is down and thus avoid having each page generated with maximum allowed response time.
  • Graceful failure. Web services are out of our control so are networks, so you should plan for web services to be too slow or unavailable sometimes. It is great if you can have partial service functionality even in such circumstances.
  • Intelligent caching. Working with local data we typically cache in memory (possibly network memory) as it is normally fast enough to regenerate the data. With Web services it is slower plus you might have limits on how frequently you can call Web Services. So you might want to cache data on Disk – in files or MySQL tables. There are multiple solutions available out where. In addition to improving speed of your application caching couple be used to provide graceful failure. It is often better to show a bit stale data rather than show no data at all.

There are many other interesting challenges and questions in Web Services world, so I should write more on this topic sometimes.

About Peter Zaitsev

Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.

Comments

  1. Manoj says:

    Question:
    Is there a way to see how the pages that have been indexed are ranked without having to type it in at Google and then start searching ?

Speak Your Mind

*