https://wiki.jenkins.io/display/JENKINS/Grant+Cancel+Permission+for+user+and+group+that+have+Build+permission

NIO and the Servlet API

From Gregw
[http://blogs.webtide.com/gregw/2004/02/09/1076359560000.html]
=============================================================

Taylor Crown has written a short paper regarding Combining the Servlet API and NIO, which has been briefly discussed on the serverside.

NIO Servlets have often been discussed as the holy grail of java web application performance. The promise of efficient buffers and reduced thread loads are very attractive for providing scalable 100% java web servers. Taylor writes about a mockup NIO server that he implemented which shows some of this promise.

Taylors results were not with a real Servlet container running realistic loads. But his results look promising and his approach has inspired me to try and apply it to the Jetty Servlet container.

The fundamental problem with using NIO with servlets is how to combine the non-blocking features of NIO with the blocking streams used by servlets. I have tried several times before to introduce a SocketChannelListener to Jetty, which only used non-blocking NIO semantics to manage idle connections. Connections with active requests were converted to blocking mode, assigned a thread and handled by the servlet container normally. Unfortunately, the cost of manipulating select sets and changing socket modes was vastly greater than any savings. So while this listener did go into production in a few sites, there was no significant gain in scalability and an actual loss in max throughput.

Taylor has tried a different approach, where a producer/consumer model is used to link NIO to servlets via piped streams. A single thread is responsible for reading all incoming packets and placing them in the non-blocking pipes. A pool of worker threads take jobs from a queue of connections with input and does the actual request handling. I have applied this approach to Jetty as follows:

• The PipedInputStream used by Taylor requires all data read to be copied into byte arrays. My natural loathing of data copies lead me to write a ByteBufferInputStream, which allows the NIO direct buffers to be used as the InputStream buffers and then recycled for later use.


• Taylors mock server uses direct NIO writes to copy data from a file to the response. While a great way to send static content, this is not realistic for a servlet container which must treat all content is as dynamic. Thus I wrote SocketChannelOutputStream to map a blocking OutputStream to a non-blocking SocketChannel. It works on the assumption that a write to a NIO stream will rarely return 0 bytes written. I have not well tested this assumption.


• There is no job queue in the Jetty implementation, instead requests are directly delegated to the current Jetty thread pool. The effect of this change is to reduce the thread savings. A thread is required for all simultaneous requests, which is better than a thread per connection, but not as trim as Taylors minimal set of worker threads. A medium sized thread pool is being used as a fixed size job queue.


• Taylors mock server only handled simple requests for static content, which may be handled with a simple 304 response. Thus no requests contained any content of size and neither do all responses. This is not a good test for the movements of real content that most web applications must do. The Jetty test setup is against a more realistic mix of static and dynamic content as well as a reasonable mix of POST requests with content.

This code has been written against Jetty 5.0 and is currently checked into Jetty CVS HEAD in the org.mortbay.http.nio package. So far I have not had time to really optimise or analyse the results, but early indications are that this is no silver bullet.

The initial effects of using the NIO listener is that the latency of the server under low load has doubled, and this latency gets worse with load. The maximum throughput of the server has been reduced by about 10%, but is maintained to much higher levels of load. In fact, with my current test setup I was unable to produce enough load to significantly reduce the throughput. So tecchnically at least, this has delivered on the scalability promise?

The producer/consumer model allows a trade off of some low and mid level performance in return for grace under extreme load. But you have to ask yourself, is this a reasonable trade? Do I want to offer crappy service to 10000 users, or reasonable service to 5000? To answer this, you have to consider the psychology of the users of the system.

Load generators do not have any psychology and are happy to wait out the increasing latency to the limits of the timeouts, often 30 seconds or more. But real users are not so well behaved and often have patience thresholds set well below the timeouts. Unfortunately a common user response to a slowly displaying web page is to hit the retry button, or worse still the shift retry! Having your server handle 1000 requests per second may not be such a great thing if 50% of those requests are retries from upset users.

I suspect that the producer/consumer model may be costing real quality of service in return for good technical numbers. Consider the logical extreme of the job queue within Taylors mock implementation. If sustained load is offered in excess of the level that the workers can handle, then that queue will simply grow and grow. The workers will still be operating at near their optimal throughput, but the latency of all requests served with increase until timeouts start to expire. Throughput is maintained, but well beyond the point of offering a reasonable quality of service.

Even with a limited job queue (as in the Jetty implementation), the simple producer/consumer model suffers from the inability to target resources to where they are best used. The single producer thread gives equal effort towards handling new requests as it does to receiving packets for requests that have already started processing. On a loaded server, it is better to use your resources to clear existing requests so that their resources may be freed for other requests. On a multi-CPU machine, it will be a significant restriction to only allow a single CPU to perform any IO reads, as other CPUs may be delayed from doing useful work or real requests, while one CPU is reading more load onto the system.

Taylors producer/consumers approach is significantly better than my preceding attempts, but has not produced an easy win when applied to a real Servlet container. I am also concerned that the analysis has focused too much on throughput without any due consideration for latency and QOS. This is not to say that this is a dead end. Just that more thought and effort are required if producer/consumer NIO is to match the wonderful job that modern JVMs do with threading.

I plan to leave the SocketChannelListener in the development branch of Jetty for some time to allow further experimentation and analysis. However, I fear that the true benefits of NIO will not be available to java web applications until we look at an API other than Servlets for our content generation.

?? 음..음..??^^ 영어.... ??



This post is part of our ReadWriteCloud channel, which is dedicated to covering virtualization and cloud computing. The channel is sponsored by Intel and VMware. Read their latest case study: A Canadian Printer Leaves its Servers in Illinois.

here was a recent post on ReadWriteCloud about 10 common mistakes made by API providers. I think this is a very thoughtful post, but I think it's an inward look at the problem. In other words, they are looking at problems that developers face while implementing their own APIs. I think biggest mistakes are not necessarily how to implement your API, but how API consumers will perceive, implement and use the API. So I came up with my own list based on nearly a decade of implementing APIs from the receiving end.

Guest author Marcelo Calbucci is the Chief Startup Officer at Conceivian, a Seattle-based Startup Lab, and the founder of Seattle 2.0, an organization providing resources for tech entrepreneurs and startups. Marcelo is also a software developer since the age of 12. You can follow him @calbucci.

1) Naming Convention

Naming convention in the software world is a debate as old as the first programming language was invented. Independent of which convention you use, be consistent. It's very annoying to implement an API that uses lowercase for all XML elements and attributes, except for a couple of them. Any developer can tell you stories of countless hours chasing a bug because of case-mismatch. My preference is all lowercase because it helps with HTTP compression.

2) URL Consistencies

This is similar to naming convention, and it's just too common to see APIs where the URL stems and query string have not a strong logic to it. As in, to get users use api.mydomain.com/getusers.php, but to get pictures use api.mydomain.com/pictures/get.php. A little bit of thought goes a long way, even if you are starting with a single API. Thinking as "objects" and "actions" is a good start.

3) Authentication

I love APIs that just ask me to pass a single API-Key on the URL. It's much simpler than having to do some digest authentication (although digest is simple too), and a lot of heck simpler than having to do a separate call to get a session cookie to use on subsequent calls.

4) Simplicity

Some engineers over-think and over-engineer the problem. YouTube used to have a beautifully simple API where you could get meta-data from a YouTube video. Now they decided to normalize and use the Google standard ATOM response, which is pretty awful and bloated. Awful and bloated is one of the reasons SOAP has not caught up. Trying to create a common response for all APIs is silly. Getting user information is different from getting a video information, which is different from posting a comment, which is different from getting a list of followers. Keep it obvious. Keep it simple.

5) Object normalization

If you are going to return an object of type User on the XML element (or JSON) then make sure that every API that returns the element is consistent and returns similar fields. It's very hard when the same provider has three different APIs that might return a similar, but not the same object. It means I have to parse it three different ways. Making attributes/values optional is fine, but don't overload their meaning.

6) Documentation

This is the most awful part of implementing APIs, particularly for newly released APIs. Don't make me figure out how each call, each element and each attribute works. I spend way too much time looking into many responses to see what's optional, what's not, what's the date format, is it a string or an integer, is it long-lat or lat-long, etc. It doesn't have to be an extensive MSDN-like documentation, but clearly stating what are the call parameters, what values are valid, what are the default values, and on the response side giving an XML-response example and describing what each element/attribute is.

7) Be Forward and Backward Thoughtful

Don't break my code! There is nothing worse when using a third-party API to learn that your live production code stopped working because the provider changed how an API works. It can be as simple as a change on the format of an element or sometimes as bad as a new XML format that is completely different from the previous format. I know you wrote on your blog, told on your Twitter account and, maybe, emailed everyone you could about this change, but don't assume people pay attention to that. The best way is to make sure the URL itself has versioning, as in api.mydomain.com/1/myapi.xml. Be committed to keep old versions for at least six months after you release a new version and be very proactive at alerting consumers of your API.

8) Error Messages Are Important

There are two points I want to make: First, "Internal Error" is not a satisfactory error message, and, second, don't overload the meaning of HTTP response status codes. The best error messages have both an English description of what they are and a parser-friendly code, as in "783". I don't want to parse English-language error messages to figure out what I should tell my user. A standard error code to indicate the Full-Name field must be present is much better. Now, we might get into preferences now, but I prefer every HTTP call to respond with status code 200 and error messages to be embedded inside of the response body (or on an HTTP header), but using 401, 403, 500 to indicate success or error is just confusing. The HTTP status code is not supposed to understand the semantic inside of the response.

9) Making it Parsing Friendly

It's important to remember as an API provider, that the cost of generating the output might be different from the cost of receiving that output and converting into a usable data structure. This goes both for the computational cost and for the implementation (lines of code) cost. Stay clear of custom date-time formats, stay clear of creating custom data types (for example, multiple pieces of information concatenated into a single string, e.g. "374|Mike Wallace|yes"). It also means don't get too creative of embedding a CSV file inside a JSON inside an XML.

10) Allow for Subsets

I really like when APIs give me the ability of choosing a subset of data on the response. Amazon e-commerce API supports that and it's great because if you just need a few fields back why would you get a 30Kb response? Depending on how I'm using the API, the CPU, network and monetary costs can be significant for me. I might also have some of the data cached, so returning a "user-id" might be enough instead of returning a full "user" object.

I don't think this is an extensive list of best practices of implementing your own API for others to consume, but I think the more you wear the hat of the consumer side of things, the more adopted your API will be. Before you even start implementing your API think of three or four applications that would be built using it, and understand what the needs would be. Maybe you should go ahead and use a bit of test-driven development, on this case usage-driven development, and implement those applications first.

Photo by m0php

Lucene : http://lucene.egloos.com/