Advanced search

Forums : Technical Support : WU Resend/Pending Question
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Paratima
Avatar

Send message
Joined: 27 Jul 07
Posts: 74
Credit: 1,285,089
RAC: 0
Message 2191 - Posted: 22 Aug 2007, 14:09:30 UTC

Now THAT is exactly what I want to know! Thank you, JRenkar.

It has already been proved that without HR enabled there becomes a LOT of invalid results.


With that sort of knowledge, I assume you must be a scientist on the project. Invalid results come from which combinations of CPU's and OS's? How do we know they are invalid? Will there be an automated method of applying corrective factors from those machines or will those results just be thrown out?
ID: 2191 · Report as offensive
zombie67 [MM]
Avatar

Send message
Joined: 25 Jun 07
Posts: 77
Credit: 5,001,188
RAC: 0
Message 2193 - Posted: 22 Aug 2007, 14:28:51 UTC - in response to Message 2191.  

Invalid results come from which combinations of CPU's and OS's? How do we know they are invalid? Will there be an automated method of applying corrective factors from those machines or will those results just be thrown out?

They are considered invalid because they don't match. If OS/CPU combo #1 returns a answer of 10, and OS/CPU combo #2 returns an answer of 20, they don't match. So a third result is issued If it goes to another OS/CPU combo #2, an answer of 20 is returned. Because the two 20s match, they are marked valid and awarded the points. The result that returned 10 is marked invalid, and awarded 0 points.

This is just a gross example, but you get the idea. When a very high level of matching is required a project, HR is sometimes the only way to get there.

FWIW, I am not sure the problem with pending results is being caused by HR. There are some fairly small projects out there that use HR, and do not have this problem. For example Docking@Home (who should be back up in a month or so, I think). They have a similar number of machines attached as Cosmology@Home, but have not had this problem.
Dublin, CA
Team SETI.USA
ID: 2193 · Report as offensive
Paratima
Avatar

Send message
Joined: 27 Jul 07
Posts: 74
Credit: 1,285,089
RAC: 0
Message 2194 - Posted: 22 Aug 2007, 14:30:46 UTC

OhioMike, you seem to have a handle on things. I understand about result X 1,000.

What I know is that engineers, programmers and scientists tend to get so wrapped up in Solving The Problem that they forget to ask if it's the right problem. Urgent is NOT the same as important. That's why we have the two different words.

I just want to be sure that my massive fleet of computers is going to be put to good use before I commit any more.

Can someone please point me to a scientific article that shows how HR is actually used to do something useful with varying results? I promise to shut up.
ID: 2194 · Report as offensive
Paratima
Avatar

Send message
Joined: 27 Jul 07
Posts: 74
Credit: 1,285,089
RAC: 0
Message 2195 - Posted: 22 Aug 2007, 14:33:16 UTC

zombie67, we have crossed in the post.

I understand what you said. I'm trying to figure out if a consensus result is necessarily correct.
ID: 2195 · Report as offensive
Profile [BAT] tutta55
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 7 Jun 07
Posts: 217
Credit: 710,406
RAC: 0
Message 2196 - Posted: 22 Aug 2007, 14:37:35 UTC

I asked a similar question some time ago in the Leiden Classical forum: If your calculations differ from one cpu to another, then how do you know which is correct? The answer is that in many simulations, there is no right and wrong. There are only different paths that are followed, and the outcome of each path is just as useful from scientific point of view. It's not the same as calculating a prime, where the outcome must be Yes or No, independent of the architecture used for calculation.

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair
ID: 2196 · Report as offensive
Paratima
Avatar

Send message
Joined: 27 Jul 07
Posts: 74
Credit: 1,285,089
RAC: 0
Message 2197 - Posted: 22 Aug 2007, 14:42:20 UTC
Last modified: 22 Aug 2007, 14:43:58 UTC

That makes my head hurt. I have to go earn a living. Until later...
ID: 2197 · Report as offensive
Profile ohiomike
Avatar

Send message
Joined: 17 Jul 07
Posts: 302
Credit: 5,006,319
RAC: 0
Message 2198 - Posted: 22 Aug 2007, 14:49:32 UTC - in response to Message 2193.  

Invalid results come from which combinations of CPU's and OS's? How do we know they are invalid? Will there be an automated method of applying corrective factors from those machines or will those results just be thrown out?

They are considered invalid because they don't match. If OS/CPU combo #1 returns a answer of 10, and OS/CPU combo #2 returns an answer of 20, they don't match. So a third result is issued If it goes to another OS/CPU combo #2, an answer of 20 is returned. Because the two 20s match, they are marked valid and awarded the points. The result that returned 10 is marked invalid, and awarded 0 points.

This is just a gross example, but you get the idea. When a very high level of matching is required a project, HR is sometimes the only way to get there.

FWIW, I am not sure the problem with pending results is being caused by HR. There are some fairly small projects out there that use HR, and do not have this problem. For example Docking@Home (who should be back up in a month or so, I think). They have a similar number of machines attached as Cosmology@Home, but have not had this problem.


One thing that has not been discussed is the validator it's self. I know that Einstein had problems with validation and problems with using HR. In the end they relaxed the validator rules to allow for the differing results. I am not sure if this is possible on this project of not however. That would have to be a question for Ben or one of the project scientists.


Boinc Button Abuser In Training >My Shrubbers<
ID: 2198 · Report as offensive
Profile [BAT] tutta55
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 7 Jun 07
Posts: 217
Credit: 710,406
RAC: 0
Message 2199 - Posted: 22 Aug 2007, 14:53:14 UTC - in response to Message 2197.  

That makes my head hurt. I have to go earn a living. Until later...


Sorry about that ;) I will try to use an analogy. Suppose you have 10 patches of land, each 10x10 meter, and you want to grow corn on it. On each you put just as many seeds and fertilizer. Then look at the end of the season what has happened. The crop on each will be similar, but not entirely the same. Next season you use more fertilizer. At the end of the season, the crop will probably differ from the season before.

That can be compared with computer similations with variable parameters. Given the same initial values for these parameters, simulations on different platforms are likely to give comparable but not necessarily equal results. What you want to study is the influence of the initial parameters (our fertilizer) on the overall outcome of the simulation.

Now, I don't know enough of the internals of this application to know if that is an entirely valid comparison. But for many simulations it is.

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair
ID: 2199 · Report as offensive
rbpeake

Send message
Joined: 27 Jun 07
Posts: 118
Credit: 61,883
RAC: 0
Message 2200 - Posted: 22 Aug 2007, 14:55:55 UTC - in response to Message 2194.  

Can someone please point me to a scientific article that shows how HR is actually used to do something useful with varying results? I promise to shut up.


I had questions on HR, too, and here is what I came up with in terms of more information:

From the BOINC site;

An interesting article from CERN, seeming to indicate how they were able to get around HR and still get the extremely consistent results they needed!

ID: 2200 · Report as offensive
Profile Jayargh
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 25 Jun 07
Posts: 508
Credit: 2,282,158
RAC: 0
Message 2201 - Posted: 22 Aug 2007, 15:05:16 UTC - in response to Message 2191.  

Now THAT is exactly what I want to know! Thank you, JRenkar.

It has already been proved that without HR enabled there becomes a LOT of invalid results.


With that sort of knowledge, I assume you must be a scientist on the project. Invalid results come from which combinations of CPU's and OS's? How do we know they are invalid? Will there be an automated method of applying corrective factors from those machines or will those results just be thrown out?



LOL You are a snippy one....I know because I have been on the project a month longer than you and that IS what occurred. Please keep a more civil tone less condescending please.
ID: 2201 · Report as offensive
Profile Angus

Send message
Joined: 28 Jul 07
Posts: 71
Credit: 31,250
RAC: 0
Message 2203 - Posted: 22 Aug 2007, 17:58:51 UTC - in response to Message 2193.  
Last modified: 22 Aug 2007, 17:59:31 UTC

They are considered invalid because they don't match. If OS/CPU combo #1 returns a answer of 10, and OS/CPU combo #2 returns an answer of 20, they don't match. So a third result is issued If it goes to another OS/CPU combo #2, an answer of 20 is returned. Because the two 20s match, they are marked valid and awarded the points. The result that returned 10 is marked invalid, and awarded 0 points.


In your example, how do we know that #1's answer of 10 is wrong? Maybe it's right, and 20 is wrong? Just because 2 PCs come up with the same answer, it doesn't mean it's the correct answer.

How far off are the answers from different CPU/OS combos? Is it just a little off, or grossly off? What do the scientists use to determine the correct answer? A quorum of 2, 3 or 10 doesn't make it correct, just repeatable.

I think the validator needs the work, and ditch HR.

ID: 2203 · Report as offensive
Profile [BAT] tutta55
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 7 Jun 07
Posts: 217
Credit: 710,406
RAC: 0
Message 2204 - Posted: 22 Aug 2007, 18:09:40 UTC - in response to Message 2203.  

They are considered invalid because they don't match. If OS/CPU combo #1 returns a answer of 10, and OS/CPU combo #2 returns an answer of 20, they don't match. So a third result is issued If it goes to another OS/CPU combo #2, an answer of 20 is returned. Because the two 20s match, they are marked valid and awarded the points. The result that returned 10 is marked invalid, and awarded 0 points.


In your example, how do we know that #1's answer of 10 is wrong? Maybe it's right, and 20 is wrong? Just because 2 PCs come up with the same answer, it doesn't mean it's the correct answer.

How far off are the answers from different CPU/OS combos? Is it just a little off, or grossly off? What do the scientists use to determine the correct answer? A quorum of 2, 3 or 10 doesn't make it correct, just repeatable.

I think the validator needs the work, and ditch HR.


Angus, please read my previous posts in here about simulations, correctness of differing results and their scientific validity.

I propose that we leave Scott a chance to comment on the posts in this thread. The thread for the moment isn't going anywhere except a HR is good/HR is bad discussion.

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair
ID: 2204 · Report as offensive
Profile Angus

Send message
Joined: 28 Jul 07
Posts: 71
Credit: 31,250
RAC: 0
Message 2205 - Posted: 22 Aug 2007, 18:31:08 UTC - in response to Message 2204.  
Last modified: 22 Aug 2007, 18:34:26 UTC

They are considered invalid because they don't match. If OS/CPU combo #1 returns a answer of 10, and OS/CPU combo #2 returns an answer of 20, they don't match. So a third result is issued If it goes to another OS/CPU combo #2, an answer of 20 is returned. Because the two 20s match, they are marked valid and awarded the points. The result that returned 10 is marked invalid, and awarded 0 points.


In your example, how do we know that #1's answer of 10 is wrong? Maybe it's right, and 20 is wrong? Just because 2 PCs come up with the same answer, it doesn't mean it's the correct answer.

How far off are the answers from different CPU/OS combos? Is it just a little off, or grossly off? What do the scientists use to determine the correct answer? A quorum of 2, 3 or 10 doesn't make it correct, just repeatable.

I think the validator needs the work, and ditch HR.


Angus, please read my previous posts in here about simulations, correctness of differing results and their scientific validity.

I propose that we leave Scott a chance to comment on the posts in this thread. The thread for the moment isn't going anywhere except a HR is good/HR is bad discussion.


I don't see anything specific to this project. Just a general analogy about land and fertilizer. Nothing about OS/CPU combos and how far they differ in their results.

Perhaps if the WUs are sent to different OS/CPU combos, and they return different results, the same WUs should be sent to a third OS/CPU combo? But then, what happens when you get a third (and different) answer?

Striving only for repeatability without regards to correctness seems to be nothing but an exercise in math computation, and not real science.





ID: 2205 · Report as offensive
rbpeake

Send message
Joined: 27 Jun 07
Posts: 118
Credit: 61,883
RAC: 0
Message 2206 - Posted: 22 Aug 2007, 19:00:24 UTC - in response to Message 2205.  

I don't see anything specific to this project. Just a general analogy about land and fertilizer. Nothing about OS/CPU combos and how far they differ in their results.

Perhaps if the WUs are sent to different OS/CPU combos, and they return different results, the same WUs should be sent to a third OS/CPU combo? But then, what happens when you get a third (and different) answer?

Striving only for repeatability without regards to correctness seems to be nothing but an exercise in math computation, and not real science.

Perhaps the link to the CERN scientific paper I posted here just a little while ago may shed some scientific light on it, perhaps answer some questions.

ID: 2206 · Report as offensive
Profile [BAT] tutta55
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 7 Jun 07
Posts: 217
Credit: 710,406
RAC: 0
Message 2207 - Posted: 22 Aug 2007, 19:01:17 UTC - in response to Message 2205.  

Perhaps if the WUs are sent to different OS/CPU combos, and they return different results, the same WUs should be sent to a third OS/CPU combo? But then, what happens when you get a third (and different) answer?

Striving only for repeatability without regards to correctness seems to be nothing but an exercise in math computation, and not real science.


As I said earlier, this third differing result would be just as valid as the 2 others. It also makes a simulation of a situation with a set of parameters. Its result would be as scientifically valid as the 2 others. Simulations don't have as goal to reach 1 specific outcome, like a yes or no answer. Only problem is, a Boinc validator has no easy way to find out which of the results are correct, or which ones have gone bad through a programming error, or which ones are faked by a user, unless it can be compared for perfect equality with another result. Hence for the moment the need for HR.

I repeat myself from the previous post, let's give Scott a chance to comment on this. All we can do is speculate, and let tensions grow as seen earlier in this thread. My task to avoid that in Scott's absence.

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair
ID: 2207 · Report as offensive
STE\/E
Volunteer tester

Send message
Joined: 12 Jun 07
Posts: 375
Credit: 16,539,257
RAC: 0
Message 2208 - Posted: 22 Aug 2007, 20:00:46 UTC

I repeat myself from the previous post, let's give Scott a chance to comment on this. All we can do is speculate, and let tensions grow as seen earlier in this thread.


Good Idea, since theres a lot of confusion right now. It's also causing more people to drop from the Project either Permanently or Temporarily until the Dust Settles. I see it right on my Team as some of the guys have already stopped or cut way back with all this turmoil going on & the Threat of not being able to get Wu's as the Supply Drys up.

I've even cut back somewhat myself in anticipation not being able to get Wu's as the availability of them drops. So I've been slowly shifting my Resource Share to other Projects so they don't go Idle.
ID: 2208 · Report as offensive
Profile Scott
Volunteer moderator
Project administrator
Project developer
Avatar

Send message
Joined: 1 Apr 07
Posts: 662
Credit: 13,742
RAC: 0
Message 2211 - Posted: 22 Aug 2007, 20:36:18 UTC
Last modified: 22 Aug 2007, 20:44:34 UTC

With so many posts by the small group of users we have thus far, I'm dreading the volume when we go to beta =)

I'll try to respond to the big issues here. Tell me if I miss anything:

Looks like your questions were totally ignored.

Is this homogenized stuff REALLY IMPORTANT? Does a 3 gigahertz Athlon 4-CPU Wondercruncher actually process the work differently than a 33MHz Pentium? Will they get different answers just by being different processors?

I'm getting a bad feeling about this project. I didn't read anywhere about this same-processor result testing when I signed on. This doesn't sound to me like a scientifically supported approach and I fear it will only continue to drag down the processing.


Yes, HR is very important (to us) for a number of reasons. Our validation scheme works by comparing two results line-by-line; the floating-point results have to agree to 1 part in 10^4 in order for agreement. You'd think that this would happen without HR anyway, but, as it turns out, around of 10% of the results fail validation without HR. Even when we're more lenient with the accuracy (say, require 1 in 10^3 precision or maybe allow something like 20 errors before a result is invalidated), the invalidation rate is still around 3-5%, which is unacceptable.

The science also benefits from HR; the results need very high accuracy to be useful. The data in the results will be fit to a curve: a variation of even 10% is enough to completely ruin the fit (i.e all the work you guys are doing would be in vain). Yes, we could get more results by not using HR, but that would result in higher invalidation (less credits for you) and lower accuracy (less reliable data for us and you).


It has already been proved that without HR enabled there becomes a LOT of invalid results.


With that sort of knowledge, I assume you must be a scientist on the project. Invalid results come from which combinations of CPU's and OS's? How do we know they are invalid? Will there be an automated method of applying corrective factors from those machines or will those results just be thrown out?

I guess you joined up after we started using HR, so you don't know about the "dark times", when 1 out of every 10 results would confer 0 credit due to invalidation. While HR begets a whole set of different problems, they are much more preferable than the alternative.

With the exception of certain Pentium CPUs (the ones with the floating point arithmetic errors, I forget which ones exactly), all OS/CPU classes will produce acceptable results. I say "acceptable" because there really is no practical way to determine which CPUs will produce the "correct" results. It might be true that a certain CPU/OS combo will always be right, or it might be that some are better for certain parameter sets than others; we really don't know. However, this source of errors can conceivably be corrected for, while the random errors produced without HR would be much much more difficult, if not impossible, to track.


Now, the situation with HR should improve once we get more users. I don't want to make any promises, but opening up registration is an option on the table at this point. It *will* happen at the latest when we go into open beta, but it might happen sooner.
Scott Kruger
Project Administrator, Cosmology@Home
ID: 2211 · Report as offensive
Profile Scott
Volunteer moderator
Project administrator
Project developer
Avatar

Send message
Joined: 1 Apr 07
Posts: 662
Credit: 13,742
RAC: 0
Message 2219 - Posted: 22 Aug 2007, 22:06:40 UTC
Last modified: 22 Aug 2007, 22:06:56 UTC

I should say that registration is more-or-less open now. You still can't register through the BOINC client, but you can through this link (also on the main page).
Scott Kruger
Project Administrator, Cosmology@Home
ID: 2219 · Report as offensive
STE\/E
Volunteer tester

Send message
Joined: 12 Jun 07
Posts: 375
Credit: 16,539,257
RAC: 0
Message 2220 - Posted: 22 Aug 2007, 22:48:29 UTC - in response to Message 2219.  

I should say that registration is more-or-less open now. You still can't register through the BOINC client, but you can through this link (also on the main page).


I Posted it in several spots in the AMD Team Forum, maybe a few people will see it and join up ...
ID: 2220 · Report as offensive
Profile [BAT] tutta55
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 7 Jun 07
Posts: 217
Credit: 710,406
RAC: 0
Message 2221 - Posted: 22 Aug 2007, 22:53:03 UTC - in response to Message 2220.  

I should say that registration is more-or-less open now. You still can't register through the BOINC client, but you can through this link (also on the main page).


I Posted it in several spots in the AMD Team Forum, maybe a few people will see it and join up ...


I posted the link in my team's forum and that of a befriended team (hi ;) ) as well.

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair
ID: 2221 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Forums : Technical Support : WU Resend/Pending Question