Array limits and memory allocation . . .

[ziza@klubicko.net]

Hi,

use dynamic memory alloction instead, it will eat up a lot of
memory :) so if you'll have problems you will have to split
your processing to several blocks and read from the file always
only part of data.

	char *g;

	g = (char *)malloc(SIZE*sizeof(char));
	...
	/* your algorithm */
	...
	free(g);

but I would not recomend this, istead I would really partition
data to several smaller blocks as I wrote earlier.

Cheers
          Ziza

On Sun, Jan 30, 2005 at 04:32:45PM -0200, Daniel Silvestre wrote:
> Greetings GCC people,
> 
> Thank you all for the helpfull tips. The problem really was the system
> size limit of the stack. I've just used the ulimit and the problem was
> gone.
> By the way, I don't know how to use memory allocation to create very
> large arrays (> 10^6 positions). My programs are very simple. But I've
> to load a entire genome/chromosome in a array. And the genome is just
> a very long string (at most 250000000 of chars).
> Some weeks ago, I wrote some perl scripts to do the same task but
> they're much more slow then the C versions. There's a code example.
> It's really a silly code so any comments will be very helpfull.
> 
> Best regards,
> -- 
> Daniel Marques
> GFT - Grupo de Física Teórica
> IFSC - Instituto de Física de S?o Carlos
> sala 16 - Edifício central
> tel: +55(16)33759769 ramal 25

> #include <stdio.h>
> #include <string.h>
> #include <stdlib.h>
> #define SIZE 250000000
> #define N 10000000
> 
> int main(int argc, char *argv[]){
> 
> 	FILE *gen, *arq1, *fdp;
> 	
> 	char *p, str[2], strng[80], g[SIZE], w[SIZE], c;
> 	int key, i, j, count, cnt, counter[N+1], sum;
> 	int counts[N];
> 	int p0, p1, p2, index, cmp;
> 	
> 	p0 = 1; p1 = 4; p2 = 16;
> 	count = 0;
> 	
> 	
> 	gen = fopen("ltrnsgn.txt", "r");
> 	while(!feof(gen)) {
> 		if((p = fgets(str,2, gen)) != NULL){
> 		g[count] = *p;
> 		count++;
> 		}
> 	}
> 	fclose(gen);
> 	
> 	for(i = 0; i < count; i++){ 
> 		
> 		if(g[i] == 'A') g[i] = 0;
> 		else if(g[i] == 'T') g[i] = 1;
> 		else if(g[i] == 'G') g[i] = 2;
> 		else if(g[i] == 'C') g[i] = 3;
> 
> 	}
> 	
> 	for(index = 0; index < 64; index++){
> 	
> 		i = j = cnt = sum = 0;
> 		for(i = 0; i < N; i++){counts[i] = 0; counter[i] = 0;}
> 		for(i = 0; i < count-2; i++) w[i] = 0;
> 	
> 		for(i = 0; i < count-2; i++){
> 			cmp = g[0+i]*p0+g[1+i]*p1+g[2+i]*p2;
> 			if(index == cmp) w[i] = 1;
> 			else w[i] = 0;
> 			
> 		}
> 	
> 		for(i = 0; i < count-2; i++){
> 			if(w[i] == 1){
> 				cnt = cnt++;
> 				j++;
> 			}
> 			else if(w[i] == 0){
> 				counts[j] = counts[j]++;
> 			}
> 		}
> 		for(i = 0; i < count-2; i++){
> 			 counts[i] =  counts[i] - 2;
> 			 if(counts[i] < 0) counts[i] = 0;
> 		}
> 		
> 		for(i = 0; i < N; i++){
> 			key = 1;
> 			j = 1;
> 			while(key == 1){
> 				if(counts[i] < j){counter[j] = counter[j]++; key = 0;
> 				}else{j++; key = 1;}
> 			}
> 		}
> 	
> 		for(i = 0; i < N; i++) sum += counter[i];
> 	
> 		sprintf(strng, "all3times%d.dat", index);
> 		
> 		arq1 = fopen(strng, "w");
> 	
> 		for(i = 0; i < N; i++){
> 			if(counter[i]){
> 				fprintf(arq1, "%d %10.9f\n", i, (float)counter[i]/(float)(sum));
> 			}
> 		}
> 	
> 		fclose(arq1);
> 	}
> 
> 	return (0);
> }

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <https://gcc.gnu.org/pipermail/gcc-help/attachments/20050131/ab6cefbe/attachment-0001.sig>