Improving our articles with help from the IT community
by Liz van Dijk on January 9, 2009 12:00 AM EST- Posted in
- IT Computing general
As some of our regular readers might have noticed, at Anandtech IT, we have been trying to spice up our articles and benchmarks by taking them a bit further than the usual rundown of standardized tests.
Instead of limiting ourselves to tests that can be found anywhere on the net, we are always eager to figure out how an application or technology will perform in a real-world situation, since in the end, that is what really matters. To this effect, we use a range of applications in combination with our in-house developed suite of benchmarking software (vApus), to get an idea of how a real application would interact with and benefit from whichever technology we're testing.
This is a strategy we have adopted during our lab work for several small to medium enterprises (SME's), since these companies usually want to see clearcut results in realistic situations and solutions they can apply for immediate profit. Giving them the opportunity of being able to test their very own specific application on their very own specific hardware has allowed us to provide them with the most relevant results possible, while giving us access to a plethora of software types that stress every part of a server.
As our IT community grows here at Anandtech, now and then we'd like to see some input for our articles as well. For example: One of our upcoming pieces will be a comparison between the possibly underestimated container-based virtualization solution and the popular hypervisor-based one. To make sure that we're not simply comparing apples and oranges, we will be performing tests with several types of applications on different VM-densities. Though we have several reliable database tests laying around, we're currently still looking for a solid, "realistic" web application to run on a Linux-based system. For it to be usable in our testing setup, we would prefer the actual web application to be mostly autonomous (e.g. backed by a database, but independent from a host of other external services), but large enough for there to be varied usage patterns.
Rather than just coming up with our own, we would like to learn from your experiences as well. If you, as IT professional, have any suggestions that might point us in the right direction, or happen to have the perfect site for us to test on hand, feel free to post a comment with your ideas or mail me directly. Our goal is to turn the IT section of Anandtech into a truly professional community for IT'ers, where contributions of regulars help set the course of our research and journalism. Now is your chance to help us start that dynamic!
Facebook
Twitter
RSS
14 Comments
View All Comments
Joe Schmoe - Wednesday, January 28, 2009 - link
I keep trying to register for the forum but I think the confirmation email is blocked by my ISP. So, then I have to use an alternate email address. Of course now my favorite username is taken by my other email address. I can never remember my login or what address used to get it.Joe Schmoe - Tuesday, January 27, 2009 - link
I know its not a very spicy topic. I'm just about ready to build or buy a new server. Its primary job will be to display a static web pages, light forms, and some email work.I'm currently running a linux box with P3 800 with 512mb of ram.
We have 2 18gig IBM SCSI drives in a raid 1 setup. I just recently had to buy one from a liquidator as a spare. They're not made anymore so I'm getting nervous.
I'm hoping the new server will have bigger hard drives because office 2007 files are too big for the mail queue. I make all the staff convert them to pdfs or zip them.
It had a redundant power supply but one side is blown. The manufacture was a vendor for gateway and I can't seem to find a phone number since Gateway is out of business. Apparently the one side has been blown for years.
It has a travan drive for backups its a travan 5 cdw says they can get me the tapes but there backorder right now. The newer tapes can't be used.
This is the primary email server for a staff of 275 and hosts 3 web sites. We process around 3000 emails per day and run a TNS system through it.
I think I might have finally convinved admin to budget for a new one.
Any suggestions?
anaz - Wednesday, January 21, 2009 - link
Hi Vaibhavi,There are several online resources available that you just google for. If any of your team like to read then quality books from wiley and oreilly cover such technologies in detail.
I also highly recommend you could attend the upcoming Great Indian Developer Summit (http://www.developersummit.com)">http://www.developersummit.com) that is covering Java, Agile, REST, JAX-RS, mashups, .NET, Rich Web, JPA, SOA, rich user experiences, Spring, Groovy and more. They have most of the creators of these technologies as speakers. My team is attending this summit 22-25 apr at IISc campus where we are attending the web conference on April 23 and java on April 24. We have been able to get very good discounts. Maybe all those who are interested from your group can sign up together and get a good bargain from them. what say? I also attended last year's conference and had a really cool time.
In Hyderabad there is Sun Tech Days with some sun speakers.
Thanks,
Anaz
Vaibhavi - Wednesday, January 21, 2009 - link
Hi all,I am working for a software integrator company. My projects includes working on Java
and Ruby on Rails and Ajax. I think Web Services is really cool. We also recently
have to now work on REST and they are talking about mashups and Struts. Can
anyone tell me if there are some good training or conferences so that me and my team
members can get to speed with these technologies. Learning from books is not my
cup of tea, even not when I was doing engineering ;)
All the help that group members can provide in this regard is much appreciated.
Thanks,
Vaibhaivi
has407 - Wednesday, January 14, 2009 - link
1. Increase your time horizon. Knowing the performance of the latest and greatest may be important, but most shops are sitting on stuff that's 2-3 years old. An important data point is how the new compares to the old. (Or to answer management's question: what does the additional $ get us vs. what we have now? Why should we spend $ to upgrade?)2. Build a representative environment and document it. A new large array of 15K SAS DAS may be appropriate for determining absolute performance--and data obtained from such configurations may be important for analysis and identifying specific strengths and weaknesses--but how many environments actually have that dedicated to any given application? More likely, there's a network/SAN/fabric in the mix. Moreover, while increasing the performance of a given component will generally increase the system's performance, that's not always the case (and when those expectations aren't as we expect, it's time to dig in and do some serious analysis).
3. Increase the scope of your metrics. Raw performance is only one part of the equation. Install, deploy, manage and maintain are equally important. While that's a much more difficult evaluation, it's extremely important for most IT shops today. A simple starting metric might be documenting how much time it takes you to configure your environment to test a new widget.
4. In short, I suggest focusing more on the environment/system, how components (hardware or software) perform in that environment, and whether the component performance is consistent with expected overall performance and that is in line with expectations. (Hmmm... component performance increased by 2.0X but overall performance increased by only 0.1X... why?) In other words, help me translate--or at least provide insight--into how Anandtech's results relate to my environment.
WillR - Wednesday, January 14, 2009 - link
Proofread. AnandTech isn't near as bad as a couple other Tech/Review sites that I won't mention, but please keep up that quality and improve whenever possible. I'm not referring to just spelling and grammar, but also making sure the site is working as it should. The way I even got to this blog post is a perfect example.In all the column clutter, under "AnandTech Blogs" I clicked the link for "Personal Experiences With The Need For NAS and SSDs" (http://anandtech.com/weblog/default.aspx#535)">http://anandtech.com/weblog/default.aspx#535) and the name anchor 535 did not exist. Thus, I ended up at the top of 19 pages of blog posts with this one being at the top as it's the latest. The timing could not have been more convenient.
As for producing better real world benchmarks, I am no help there. But I would like to ask why so many interesting hardware products are overlooked... by everyone. I've tried and failed at finding a comparison of the Xeon L54XX CPUs to anything. Also it would be nice to see tests of drives other than the flagship sizes. The WD RE3 drives have been out for over 6 months, and no reputable review site has tested either single platter models like the WD3202ABYS. They should be quieter, use less power, and less prone to failure due to fewer parts compared to the 1TB models. Many user forums have threads talking about them but the test results, while impressive, are meaningless since the test parameters are not consistent.
DougG - Sunday, January 11, 2009 - link
I like the approach you're taking of finding representative apps to benchmark.I'd like to toss in an app suggestion: Hyperic HQ. It's an open source monitoring and data collection app with a Java J2EE backend, and an integrated PostgreSQL database.
Hopefully it would be fairly easy to set it up in some kind of load testing mode where data is fed in as fast as the system can handle it.
LizVD - Monday, January 12, 2009 - link
Thanks alot for this feedback, this is the sort of stuff we need! :)I'll check this out right now, it looks very interesting.
Alpha2Zeta - Saturday, January 10, 2009 - link
You all need to keep up-to-date with new x264 revisions since the latest versions of x264 have Core i7 optimizations and will soon have Phenom II optimizations.Graysky's x264 HD test uses an old version of x264.
Also, please state the x264 settings used in those benchmarks, different settings can have very different fps results.
Try one of MeGUI's preset profiles, like the PS3 profile, if you're looking for well thought out x264 settings.
And remember, CRF is the norm now, not 2-pass!
JarredWalton - Tuesday, January 13, 2009 - link
While x264 is great for video encoding, I just don't know that it's particularly relevant to IT work. It's more of an underground "hacker/pirate" thing in my experience; most businesses are primarily focused on using existing video technology, or the standardized H.264 codecs used in Blu-ray.