Value driven web development

Building Ferret queries in models and ferret pagination

August 28th, 2007

With the help of acts_as_ferret plugin, it is very easy to add ferret search functionalities to active records. However, it doesn’t come with pagination out of the box. Following the good practise of fat models and skinny controllers, I’ll show you how I structure my search methods with ferret pagination.

First – get the pagination find plugin. Ilya released a very useful script to combine with paginationfind, however it is not using exactly the same syntax as paginationfind and there’s a tiny issue with the results count when active record :conditions are included.

I’ve created my own patched version here, it accepts the same :page => { :size => 10, :current => 1 } hash.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

def paginating_ferret_search(query,options = {}, find_options = {})

  page_options = find_options.delete(:page) || options.delete(:page) || {}

  current = page_options[:current] && page_options[:current].to_i > 0 ? page_options[:current] : 1
  first = page_options[:first] || 1
  auto = page_options[:auto] || false

  # Total size is either AR find result count or Search result total_hits or from limit param, whichever is less
  limit = options.delete(:limit)
  total_hits = if find_options[:group] || find_options[:conditions] || find_options[:select]
                  ids = []
                  raw_hits = find_id_by_contents(query, {:limit => :all})   {|type, id, score, data_hash| 
                     ids << id 
                  }
                  conditions_array = combine_conditions([ "#{self.table_name}.#{self.primary_key} in (?)", ids ], find_options[:conditions])
                  count :all, find_options.update({:conditions => conditions_array})        
                 #find_by_contents(query, {:limit => :all},find_options).total_hits # <- bad way of doing it     
               else
                 find_by_contents(query, {:lazy => true}).total_hits      
               end
  total_size = limit ? [limit, total_hits].min : total_hits


  # If :size isn't specified, then use the lesser of the total_size
  # and the default page size
  page_size = page_options[:size] || [total_size, DEFAULT_PAGE_SIZE].min


  PagingEnumerator.new(page_size, total_size, auto, current, first) do |page|

    # Set appropriate :offset and :limit options for this page
    offset = {:offset => (page - 1) * page_size  }
    limit = {:limit => (page_size) < total_size ? page_size : total_size}
    # set paging settings on AR results if AR find option exists.
    # otherwise set paging settings on Ferret results.
    if find_options[:group] || find_options[:conditions] || find_options[:select]
      find_options.merge!(offset).merge!(limit)
      options[:limit] = :all
    else
      options.merge!(offset).merge!(limit)
    end

    find_by_contents(query, options, find_options)
  end
end
Next in my product.rb model , I have this class method. 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

class Product < ActiveRecord::Base

def self.search(raw_query, *args)
    options = args.extract_options!
    brand_query = options.delete(:brand)

    query = Ferret::Search::BooleanQuery.new() 
    query.add_query Ferret::Search::FuzzyQuery.new("*", raw_query)
    query.add_query Ferret::Search::TermQuery.new(:brand_name, brand_query), :must unless brand_query.blank?
    query.add_query Ferret::Search::TermQuery.new(:hidden, "N"), :must if options.delete(:public) == true

    puts query.to_s
    self.search_by_ferret_query(query.to_s, options)
  end

  def self.search_by_ferret_query(query,options)
    if options[:page] 
        self.paginating_ferret_search(query, options)    
    else
        self.find_by_contents(query, options)
    end    
  end

end
Posted by admin Filed in
0 comments »

How to compile ffmpeg on Mac and Linux

August 18th, 2007

Default compile command 
1
2
3

tar zxvf xxx.tar.gz
./configure --prefix=/usr --enable-shared
make && sudo make install
1, lame 
1
2

http://sourceforge.net/project/showfiles.php?group_id=290 (lame-3.97.tar.gz)
compile
2, Ogg, Vorbis, Theora 
1
2

http://xiph.org/downloads/ (libvorbis,libtheora,libogg)
compile
3, NASM(support compile xvid and x264) 
1
2

http://sourceforge.net/project/showfiles.php?group_id=6208 (nasm sources)
compile
4, xvid http://downloads.xvid.org/downloads/ 
1
2
3

tar xvidcore-1.1.3.tar.gz
cd xvidcore-1.1.3/build/generic/
compile
Problem: nasm: fatal: unrecognised output format ../../src/bitstream/x86_asm/cbp_3dne.asm - use -hf for a list (http://rob.opendot.cl/index.php/useful-stuff/xvid-with-asm-on-os-x/) Solution: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

./configure --prefix=/usr --enable-shared --enable-macosx_module
vi platform.inc:
< SPECIFIC_CFLAGS=-fPIC -fno-common -no-cpp-precomp 
< CFLAGS=-Wall -O2 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer
---
> SPECIFIC_CFLAGS=-fPIC -fno-common -no-cpp-precomp -DHAVE_PTHREAD -undefined suppress
> CFLAGS=-Wall -O3 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer -march=prescott
41c41
< AFLAGS=-I$(<D)/ -f   
---
> add '-f macho -DPREFIX' at the end of the line
48c48
< SPECIFIC_LDFLAGS=-r -keep_private_externs -nostdlib && $(CC) $(LDFLAGS) $(PRE_SHARED_LIB) -o libxvidcore.$(SHARED_EXTENSION).$(API_MAJOR) -bundle -flat_namespace -undefined suppress 
---
> SPECIFIC_LDFLAGS=-r -keep_private_externs -nostdlib && $(CC) $(LDFLAGS) $(PRE_SHARED_LIB) -o libxvidcore.$(SHARED_EXTENSION).$(API_MAJOR) -bundle -flat_namespace -undefined
make
sudo make install
5, x264 svn co svn://svn.videolan.org/x264/trunk x264 compile Problem: common/i386/dct-a.asm:124: panic: invalid section name (for Macosx) Solution: 
1
2
3
4
5
6

install latest nasm and xcode(maybe re-install it, need some package installed)
download patch http://www.via.ecp.fr/via/ml/x264-devel/2006-12/msg00030.html
patch < ./x264-darwin-build.patch  ./configure --prefix=/usr --enable--dynamic 
make clean 
make  
sudo make install
6, a52codec http://liba52.sourceforge.net/ (a52codec-0.7.4) compile Problem: on ubuntu 
1
2

./configure --prefix=/usr 
make && sudo make install
7, mpg4 aac http://sourceforge.net/project/showfiles.php?group_id=704 (faad2-src, faac-src) 
1
2
3
4
5
6

A). cd faac
chmod +x bootstrap
./bootstrap
./configure --prefix=/usr --with-mp4v2 --enable-shared
make
sudo make install
Problem : cannot run ./bootstrap on ubuntu 
1
2
3

sudo apt-get install autoconf
sudo apt-get install automake
sudo apt-get install libtool
Problem: bad interpreter: No such file or directory. (for Macosx) Solution: 
1
2
3
4

change bootstrap to unix format (or use dos2unix)
tr -d '\015' < bootstrap > bootstrapa
mv bootstrapa bootstrap
chmod +x bootstrap
Problem: cannot find libtoolsize with Mac Solution: 


sudo ln -s /usr/bin/glibtoolize /usr/bin/libtoolize
Problem: .infig.status: error: cannot find input file: Solution: 
1
2

find . -type f | xargs dos2unix
chmod +x configure
B). cd faad2 
1
2
3
4

./bootstrap
./configure --prefix=/usr --with-mp4v2 --enable-shared
make
make install
Use the following command with Mac i386Ôºö 


make "CC=gcc -arch i386 -DHAS_LRINTF -Z -L'/Developer/SDKs/MacOSX10.4u.sdk/usr/lib'"
Problem: on ubuntu 
1
2
3

download http://packages.ubuntu.com/feisty/libs/libfaad2-0
http://packages.ubuntu.com/dapper/libdevel/libfaad2-dev
sudo dpkg -i xxx.deb
Problem: plugins/Makefile.am:3: required directory plugins/bmp does not exist Solution: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

vi plugins/Makefile.am
Delete
if HAVE_BMP
SUBDIRS = bmp
else
SUBDIRS =
endif #HAVE_BMP

vi configure.in
Delete
if test x$WITHBMP = xyes; then
 AC_DEFINE([HAVE_BMP], 1, [User wants beep media player plugin built])
 AM_CONDITIONAL([HAVE_XMMS], true)
 AM_CONDITIONAL([HAVE_BMP], true)
fi

vi plugins/xmms/src/Makefile.am
Change the following line(delete everything about HAVE_BMP)
if HAVE_BMP
libdir=$(shell pkg-config --variable=input-plugin-dir bmp)
local_CFLAGS=$(shell pkg-config --cflags bmp)
local_LDFLAGS=$(shell pkg-config --libs bmp)
else
local_CFLAGS=`$(XMMS_CONFIG) --cflags` -Wall
local_LDFLAGS=`$(XMMS_CONFIG) --libs`
libdir = `$(XMMS_CONFIG) --input-plugin-dir`
endif #HAVE_BMP
To
local_CFLAGS=`$(XMMS_CONFIG) --cflags` -Wall
local_LDFLAGS=`$(XMMS_CONFIG) --libs`
libdir = $(XMMS_CONFIG) --input-plugin-dir
Problem: /usr/include/architecture/i386/math.h:378: error: conflicting types for 'lrintf' common.h:329: error: previous definition of 'lrintf' was here Solution: 
1
2
3

vi libfaad/common.h
311 line add:  #ifndef HAS_LRINTF
339 line add:  #endif
8, 3gp http://www.penguin.cz/~utx/amr (amrnb and amrwb) compile 9, ffmpeg 
1
2
3
4
5
6
7
8

cd ffmpeg
./configure --prefix=/usr --enable-gpl --enable-pthreads --disable-ffserver --disable-ffplay --enable-pp --enable-shared --enable-libmp3lame --enable-libamr-nb  --enable-libamr-wb --enable-libogg --enable-libtheora --enable-libvorbis --enable-libxvid --enable-liba52 --enable-liba52bin  --enable-libfaad  --enable-libfaadbin --enable-libfaac --enable-libx264
make
sudo make install

ffmpeg --version
FFmpeg version SVN-rUNKNOWN, Copyright (c) 2000-2007 Fabrice Bellard, et al.
  configuration: --prefix=/usr --enable-gpl --enable-pthreads --disable-ffserver --disable-ffplay --enable-pp --enable-shared --enable-libmp3lame --enable-libamr-nb --enable-libamr-wb --enable-libogg --enable-libtheora --enable-libvorbis --enable-libxvid --enable-liba52 --enable-liba52bin --enable-libfaad --enable-libfaadbin --enable-libfaac --enable-libx264
If the ffmpeg command isn't the new one after re-compile with mac, please: 
1
2
3
4

sudo rm /usr/bin/ffmpeg
sudo rm /usr/local/bin/ffmpeg
sudo make install
sudo ln -s /usr/bin/ffmpeg /usr/local/bin/ffmpeg
./configure -h ./configure -h > ffmpeg_config.txt dos2unix, mac2unix for Mac 
1
2
3
4
5
6
7
8
9
10
11
12

wget http://fresh.t-systems-sfr.com/linux/src/dos2unix-3.1.tar.gz
tar zxvf dos2unix-3.1.tar.gz
cd dos2unix-3.1
rm -f dos2unix
rm -f dos2unix.1
rm -f mac2unix
rm -f mac2unix.1
make all
sudo install -m 755 dos2unix /usr/local/bin
sudo install -m 644 dos2unix.1 /usr/local/man/man1
sudo install -m 755 mac2unix /usr/local/bin
sudo install -m 644 mac2unix.1 /usr/local/man/man1
Posted by admin Filed in
5 comments »