[prev in list] [next in list] [prev in thread] [next in thread] 

List:       eros-arch
Subject:    Hardware for EROS?
From:       valerio bellizzomi vbnet () rm ! ats ! it
Date:       2000-06-23 20:49:30
[Download RAW message or body]

Hello,
I include the entire message because this is an old thread, I want to share
something about the third alternative discussed in this message and other
things.
Scroll the message to find my comments (which all start and end with a #
for clarity).

valerio

*********** REPLY SEPARATOR  ***********

On 11/04/2000 at 15.04 kragen@pobox.com wrote:

>Mark Miller writes:
>> I agree.  The military seems to be interested in the exotic technology
>> because they want instant-on.  I don't care about instant-on; but for
>> distributed computing, I'm desperate for low latency stable storage so
that
>> I can do instant commit.  Then, I could afford to have machine X not
release
>> message M until the state that had computed message M had been committed
to.
>> 
>> If a disk seek is the cost of a commit, then many realistic distributed
>> computing scenarios cannot wait that long before releasing a message.
>
>"low latency stable storage" can take many forms, with varying kinds of
>stability.

#
and varying kinds of low latency.
#

>
>The Rio guys, who also built Vista, just use DRAM as their "stable
>storage".  When the OS crashes, they write the Rio area to disk before
>rebooting.  They claim this works extremely well in their tests (in
>which they simulated software failures by randomly flipping bits) ---
>in fact, after they did some VM twiddling to protect the Rio area from
>random kernel writes, they claimed it was more reliable than just
>writing stuff to disk.
>
>This obviously has problems if you're on flaky hardware that can crash
>without giving the OS this chance to recover.  It also obviously has
>problems if someone can press the power button.
>
>The NetApp guys have a couple of megabytes of RAM --- can't remember
>what kind, but probably SRAM, so mucho dinero --- on a card with two
>batteries to sustain it through power failures.  They commit their
>"disk writes" to there.

#
This is ok, but if you can enforce physical security on your sites, the RAM
is more reliable than disks because there are no mechanical parts in
movement.
And in this case you don't need disks at all.
#

>
>A third alternative --- probably not reasonable if you want <10ms
>latency, but possibly reasonable if you want high throughput --- is to
>use the RAM of several geographically-separated machines connected via
>the Internet as your "stable storage".  As long as they don't all lose
>power simultaneously or close to it, and as long as you don't have any
>non-fail-stop failures that corrupt your data, this is probably more
>reliable than using a disk.

#
Provided that the machines are all powered by different geographically
distant power sources, and that the machines are all fault-tolerant (two or
more machines locally interconnected by a synchronous heart-beat LAN link,
I will call that a site), and that you use ping to check remote sites so
that if a site goes down all others immediately write image to the RAM of
all local machines, the solution may be more reliable that using disks, but
there is another solution: reflective memory.
Reflective memory is an all-hardware solution formed by memory boards
interconnected by a dedicated network. When a change is written to a board,
the change is immediately reflected to all other boards, and because the
dedicated network is attached directly to the memory boards, there is no
cpu horsepower usage by this hardware mechanism.
This solution seems to be more performant but, maybe a bit limited in
distance between sites.
#

>
>(Non-fail-stop failures that corrupt your data can happen even if you
>are using a disk.)
>
>I don't know if this has been tried; it is likely to have unforeseen
problems.
>
>If your network is fast enough, this alternative could be faster than
>using a disk.

#
Because the reflective memory boards have a dedicated network, the solution
is not only faster than the third alternative, but also more reliable
because you don't have to deal with other network problems, such as network
interface cards and router problems,and other internet problems in general
which are dependent on events that arise outside your own sites.
#

>
>
>By the way, I'm interested to hear about your distributed-computing
>scenarios that require stable commitment.
>
>-- 
><kragen@pobox.com>       Kragen Sitaker
<http://www.pobox.com/~kragen/>
>The Internet stock bubble didn't burst on 1999-11-08.  Hurrah!
><URL:http://www.pobox.com/~kragen/bubble.html>
>The power didn't go out on 2000-01-01 either.  :)

[prev in list] [next in list] [prev in thread] [next in thread]