The Joyent Community

By "older" do you mean shared accelerators that were some of the first to be deployed? If so, does it make sense to update the shared accelerator configuration or just redeploy those on the "older" setups? Do you have a sense for when these issues will be resolved? (N hours, N days, N weeks, or N months, where N<10)

I'm sure the Joyent staff has been putting in extra hours and effort to try to get to the bottom of this. THANKS!

Yay! Thanks.

madams wrote:

The only thing you need to add is an time frame for when you plan to have the solution in place.

From yesterday til it's done.

Hope this relates to bolinas and johnson. Cuz they're slower than molasses right now. I don't know many users who are willing to watch a progress bar for 30 seconds.

But I do appreciate the update. Thanks

Yes, for sure was seeing this on Myrtle and Magnolia. Great to hear you're working on in.

bretthoerner wrote:

Reid's homepage just took 1m33.681s to fetch (when it doesn't throw a 50x).

Yes, I've had the role reversal of three clients contacting me to report that my site is down. Even my 80 year old Mom wants to know what's up with my site. My niece isn't happy either. Being 2.5 years old, that's not uncommon, though.

Me? I'm calm about it only because I don't have the time to be any other way.

One more thing that wasn't mentioned on this thread was the severity of the problem.
I'd like to contribute a few statistics that I collected today.

To measure performance, I ran "time ls > /dev/null" repeatedly on a directory which contains less than 30 files. I did this on two machines (cardero and bridgeway).

Some highlights of the results:

1. Most of the time (>55%), 'ls' on cardero (FreeBSD, shared hosting) takes less than 5ms to complete.
2. On bridgeway (the Solaris Premier container), only about 16% of the commands take <5ms.
3. A very significant portion of the time (>25%), the command on bridgeway takes over 100ms to complete
4. About 1% of the time the command on bridgeway took between 10 seconds and a full minute. On 2 instances the command took over 2 minutes to execute!

Here's some detailed info on what I did. To start, I used the following little piece of bash code:

while true; do echo "----"; date; time ls > /dev/null; sleep 5; done

I collected the output, did some parsing on it using a small php script, and did a histogram analysis of the time elapsed.
I did this it on both bridgeway (shared container, Solaris, premier) and on cardero (an old FreeBSD, shared hosting machine).
Hopefully there aren't too many bugs in my analysis script ;)

The results I got were as follows:

Results for cardero (FreeBSD shared)

  Total measurements: 5288
  Measurements taken from  Thu, 29 Nov 2007 -- 09:16:04 GMT  to Thu, 29 Nov 2007 -- 20:15:31 GMT
  less than 5ms:  -- 55.862% (2954)
  5ms - 10ms:     -- 11.876% (628)
  10ms - 50ms:    -- 17.114% (905)
  50ms - 100ms:   -- 4.274% (226)
  100ms - 1sec:   -- 7.451% (394)
  1sec - 5sec:    -- 2.156% (114)
  5sec - 10sec:   -- 0.756% (40)
  10sec - 20sec:  -- 0.473% (25)
  20sec - 30sec:  -- 0.038% (2)
  30sec - 60sec:  -- 0.000% (0)
  1 - 2 minutes:  -- 0.000% (0)
  over 2 minutes: -- 0.000% (0)


    
Results for bridgeway (Solaris shared container, Premier)
  Total measurements: 7076
  Measurements taken from  Thursday, 29 November 2007 -- 9:03:54 GMT  to Thursday, 29 November 2007 -- 20:16:46 GMT
  less than 5ms:  -- 15.941% (1128)
  5ms - 10ms:     -- 7.292% (516)
  10ms - 50ms:    -- 34.483% (2440)
  50ms - 100ms:   -- 15.941% (1128)
  100ms - 1sec:   -- 20.548% (1454)
  1sec - 5sec:    -- 3.477% (246)
  5sec - 10sec:   -- 1.074% (76)
  10sec - 20sec:  -- 0.565% (40)
  20sec - 30sec:  -- 0.212% (15)
  30sec - 60sec:  -- 0.254% (18)
  1 - 2 minutes:  -- 0.184% (13)
  over 2 minutes: -- 0.028% (2)

I am extremely hopeful that 'til it's done' will be very very quick.

Last edited by ronp001 (2007-11-30 14:45:19)

I mean "L", "XL" and "XXL" hosting plans

Err, now, is this a recent problem? It's also possible that your site/app is slow regardless of this I/O problem and that you DO indeed need more CPU/RAM. A non-shared Accelerator can't hurt, for sure.

bretthoerner wrote:

Err, now, is this a recent problem? It's also possible that your site/app is slow regardless of this I/O problem and that you DO indeed need more CPU/RAM. A non-shared Accelerator can't hurt, for sure.

We have no traffic at all on our website and it loads very slow, even timeouts sometimes.

bretthoerner wrote:

we can just hope they fix it soon.

Very soon, I hope. Seems our sites have become totally unusable.

What is spindle? I guess it is some kind of hardware but what is it?

:) I took it too serious heh!

David's update is good also:
http://discuss.joyent.com/viewtopic.php … 57#p155657

As one of the people who is an IO user I appreciate that Joyent wants to fix the issue.

Thanks guys. My accelerator seems fine, but Litho, Tamalpais, and Mariposa have been having issues. I have faith.

I found a silver lining to this: Litho's recent flakiness has allowed me to reproduce some previously un-reproducible Coda bugs. All I needed was a shitty server to test on. :)

That said, let's hope this doesn't continue much longer. I've got a musical advent calendar launching soon.

Jan wrote:

I predict a performance melt down as Reid's error pages get a mention on slashdot.

Can we have a mirror? Unable to even get an error page at the moment :(

lol

Spintech page loads for me but slowly. Is that on a Shared Accel?

EDIT: I see it is on a Premiere box.

Last edited by jdjustice (2007-11-30 07:23:01)

Now your site visitors have something to look at when they visit your homepage! Wait... that is your homepage. :/

reid wrote:

jdjustice wrote:

EDIT: I just got the 503 page waiting for the home page LOL. But that one's not even remotely funny, just sad

Error 503, New & Improved: Less Sad, More Funny! But no accordion. No time for a proper illustration.

Reid's is also pretty decent!