CCS C Software and Maintenance Offers
FAQFAQ   FAQForum Help   FAQOfficial CCS Support   SearchSearch  RegisterRegister 

ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CCS does not monitor this forum on a regular basis.

Please do not post bug reports on this forum. Send them to support@ccsinfo.com

Data Storage
Goto page 1, 2  Next
 
Post new topic   Reply to topic    CCS Forum Index -> General CCS C Discussion
View previous topic :: View next topic  
Author Message
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

Data Storage
PostPosted: Sat Aug 29, 2020 12:08 pm     Reply with quote

Hello,

For now I have to store about 2000 rfid tags numbers 10 bytes long.
On my hardware I have x2 FM24W256 i2c F-RAMs, AT45DB041E SPI memory and SD Card.
Which memory will be the best choice?
I have made some tests with i2c F-RAM and when I get a new card number and have to compare if it exists, it takes about 5 sec. which is acceptable, but if cards are more (10 000)?

P.S I have no driver for AT45DB041 and SD card.

Best Regards!
newguy



Joined: 24 Jun 2004
Posts: 1899

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 12:32 pm     Reply with quote

If you swap the I2C FRAM for SPI FRAM, you should be able to cut your search time by a factor of ~10. 400kHz clock vs 5MHz+.
dluu13



Joined: 28 Sep 2018
Posts: 395
Location: Toronto, ON

View user's profile Send private message Visit poster's website

PostPosted: Sat Aug 29, 2020 12:35 pm     Reply with quote

If you use SD card and you don't want to buy a driver, mmcsd.c driver provided by CCS works after a number of tweaks. There are a bunch of sticky threads on the topic. I was never able to get fat.c working with it though.
temtronic



Joined: 01 Jul 2010
Posts: 9081
Location: Greensville,Ontario

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 1:30 pm     Reply with quote

How fast is the PIC running ?

What kind of 'compare' algorithm are you using ??

My 'gut' says 5 seconds for a search/compare is way too long. I was accessing 6000 item databases, 10 byte 'stock number' and it was a lot faster than 5 seconds on a Z80 running 2MHz clock.

Hmm, which PIC ? as one with lots of RAM and EEPROM should be faster than external I2C or SPI devices.
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 1:31 pm     Reply with quote

newguy wrote:
If you swap the I2C FRAM for SPI FRAM, you should be able to cut your search time by a factor of ~10. 400kHz clock vs 5MHz+.


Hi mr."newguy",

Because my hardware now has AT45DB041E Flash what about it?
I think it also is high-speed memory ?
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 1:38 pm     Reply with quote

dluu13 wrote:
If you use SD card and you don't want to buy a driver, mmcsd.c driver provided by CCS works after a number of tweaks. There are a bunch of sticky threads on the topic. I was never able to get fat.c working with it though.


Hi,
I have no experience with SD card but three months ago I tested CCS's MMC driver with no success. Can you point me for some stable working driver?

Thanks,
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 1:44 pm     Reply with quote

Quote:
How fast is the PIC running ?.

24Mhz
Quote:

What kind of 'compare' algorithm are you using ??

Read ten bytes and memcmp();

My 'gut' says 5 seconds for a search/compare is way too long. I was accessing 6000 item databases, 10 byte 'stock number' and it was a lot faster than 5 seconds on a Z80 running 2MHz clock.

Quote:

Hmm, which PIC ? as one with lots of RAM and EEPROM should be faster than external I2C or SPI devices
dsPic33EP512MU810
newguy



Joined: 24 Jun 2004
Posts: 1899

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 2:43 pm     Reply with quote



Last edited by newguy on Sat Aug 29, 2020 4:10 pm; edited 2 times in total
temtronic



Joined: 01 Jul 2010
Posts: 9081
Location: Greensville,Ontario

View user's profile Send private message

PostPosted: Sat Aug 29, 2020 2:55 pm     Reply with quote

re: ...
Quote:
Read ten bytes and memcmp();

so a 'linear' search ???

NOT efficient or fast....

You should do a one time sort of the 2000 ID tag 'database', then you can use a 'binary' search, which will take 10-11 compares. This will be very fast with a PIC running 24MHz.

Others who do databases in PICs can maybe show parts of their code.
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 2:38 am     Reply with quote

Lets think for a moment. A memcmp on ten bytes, should only take perhaps
50 instruction times.
Assume the 'search for' value is sitting in an array, and the 'search in'
value is read in on I2C. Reading these at 400KHz, will take 2000*10*
2.5uSec * 2(say) *8 (bit versus byte) *1.5 (address) = 1.2 seconds.
So perhaps 1.2 seconds. Then if the processor is running at (say)
32MHz (8 MIPS), the search would add an average (assuming you average
searching half the records), of perhaps a tenth of a second. So it should be
totally easy to do this in well under 1.5 seconds.
The figure of 5 seconds, suggests something is slowing this a lot. Possibly the
I2C is only being run at 100KHz, or software I2C is being used?.
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 4:07 am     Reply with quote

Hi mr."Ttelmah",

Yes Software i2c are used. Also i have i2c bus speed limits max=400 Khz because on the same bus has RTC (mcp79410).
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 6:55 am     Reply with quote

Seriously use hardware I2C.....
Even if it means changing the chip.
Software I2C, even if set for 400KHz, is likely to only be doing something
like 130KHz. Hardware will make a huge difference to the performance.
I'd expect the time to be well under half of what you currently have,
if you can switch to using the hardware.....
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 9:11 am     Reply with quote

Hi,

What will be the appropriate i2c speed when switch to hardware i2c?
(Max speed for MCP79410 = 400kHz). Pull res. = 2k at 3v3?
Is it a good idea to switch to SPI AT45DB041E? (Some working driver?)

Best Regards!
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 9:31 am     Reply with quote

Just set the I2C, to 400K. This is the maximum for standard I2C 'FAST' mode.
Yes, using an SPI chip would be a lot faster. Data transfer times perhaps
5* faster in the 'real' world. The I/O part of the driver would need
tweaking.
There are other things that could make a lot of difference though. If the data
is sequential use a 'page mode' driver for your existing chip. This will boost
the overall performance possible 50%, Removes the need to keep sending
the address to be read. These have been published here before.
kmp84



Joined: 02 Feb 2010
Posts: 345

View user's profile Send private message

PostPosted: Sun Aug 30, 2020 1:54 pm     Reply with quote

Thank you All for suggestions!
I'll try to decrease time for searching in "F-RAM DB".

Best Regards!
Display posts from previous:   
Post new topic   Reply to topic    CCS Forum Index -> General CCS C Discussion All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group