Back from holiday

Ahhh…holiday was great. Now i have 3 weeks left to finish and write up my degree project. wahey! I need to leave the video conversion business for now and get the thing online and usable.

Orginally i was hoping to get my uni to host a rails server – they seemed up for it but hadn’t done a rails server before, which made me a bit nervous – my uni is two hours travel away and i didn’t want to have to go in to try to get it up and running too often. So, i decided to go with a paid web hosting service. I’d been looking at BlueHost and a friend suggested them, so i went with them. $7.95 a month for 12 months, comes out at about £48 for the year.

I decided to change the name to www.reeplies.com, with the ’stories’ that people attach to articles renamed to ‘reeplies’ – this makes the focus of the site more on the reeplies than the articles, which is good.

Anyway, going through the bluehost setup wizard at the moment. After getting it set up i’m going to get capistrano up and running so i can update it easily.

OK, finished the wizard, now i need to set up my stuff for uploading to the server. Before i do that i’m going to do that rename of stories to ‘reeplies’. Most of it is just simple cosmetic stuff, but one thing that isn’t is the urls: instead of the url for editing a story being

www.reeplies/story/edit/84

i want it to be

www.reeplies/reeply/edit/84

This involves routes, which i’ve seen referenced but not used. To skip past all my fumbling about and forum questions, this is how i did it: in config/routes.rb, add the following line:

map.story ‘/reeply/:action/:id’, :controller => “story”

Amazingly, this is all that’s required – i don’t have to change any of my link_to or redirect_to parameters – it just works! Outstanding!

I did have one little problem that puzzled me for a while – my reeply form submission wasn’t working. This turned out to be due to a case statement that i was using to detect which button the user pressed on the form page (they could submit the form, add a picture or add a movie). The case statement works by looking at the value in params[:commit], which contains the text displayed on the buttons – the easiest/only? way to tell them apart in the way i set it up. So i had to change the case condition to use the updated button text “Submit Reeply”. Sorted.

So, i think it’s ready to get hosted now. Obviously there’s still lots of stuff i want to fix but it’s useable and has the right names for stuff. So, on with the hosting setup!

…

One day later, no progress. One of the brighton Ruby group people has kindly offered to help, a bit later in the week. So, in the meantime, i’m going to do a bit of tweaking – in particular, to let the user rate reeplies up and down. This will work pretty much the same as for articles – for example, a reeply added by a user will automatically be marked as having been scored up by that user.

So, just as we had a ’scores’ class that had user_id, article_id and points, we now need a story_scores class to do the corresponding task for stories/reeplies. (i’m calling it story_scores as reeplies are still called stories everywhere in the code). Time to go back and read my old blog post about setting up scores! Actually, on rereading that i remembered that i revised it loads as i went along, making it messy and hard to follow. So, here’s the final working version, but for stories this time.

At the command line:

ruby script/generate model StoryScore

In the new story_score.rb file:

class StoryScore < ActiveRecord::Base
belongs_to :story
belongs_to :user
end

In story.rb:

has_many :scores, :through => :story_scores
has_many :scorers, :through => :story_scores, :source => :user

A migration to make the table -

class CreateStoryScores < ActiveRecord::Migration
def self.up
create_table :story_scores do |t|
t.column :story_id, :integer
t.column :user_id, :integer
t.column :points, :integer
end
end

def self.down
drop_table :story_scores
end
end

A change_score method for story_controller:

def change_score
begin
@story = Story.find(params[:story_id])
rescue ActiveRecord::RecordNotFound
flash[:message] = “Couldn’t find Reeply##{params[:story_id]}, redirecting to article list.”
redirect_to :controller => “article”, :action => “list”
else
flash[:message] = @story.change_score(params)
redirect_to :controller => “article”, :action => “show”, :id => @story.article_id
end
end

And the story model method it calls:

def change_score(params = {})
new_points = params[:points].to_i
@score = StoryScore.find(:first, :conditions => ["story_id = ? and user_id = ? ", params[:story_id], params[:user_id]])
if @score == nil
#@score not found
#no score exists from this user so make a new one then add the points
return_string = “Didn’t find score, making a new one”
@score = StoryScore.new(:story_id => params[:story_id], :user_id => params[:user_id], :points => new_points)
self.points += new_points
@score.save
self.save
else #found something for @score in db already
if @score != nil #@score was already there
#look for a nil and turn it into a 0 if present
@score.points = 0 if @score.points == nil
if @score.points != new_points
#score already there but points are different
#subtract old points then add new points
self.points -= @score.points
self.points += new_points
return_string = “changed your score for this story from #{@score.points} to #{new_points}”
#then change @score to have new points
@score.points = new_points
@score.save
self.save
else
return_string = “you already gave this story #{@score.points} point”
end #if
end #if
end #if
return return_string
end #method

And the simple story model methods called by the above method:

def score_from_user(params = {})
begin
score = StoryScore.find(:first, :conditions => ["story_id = ? and user_id = ? ", self.id, params[:user_id]])
return score.points
rescue
return 0
end
end

def change_points(params = {})
self.points += params[:amount]
self.save
end

Cool, that all seems to be working ok! Noticed a couple of bugs though:

• Uncompleted reeplies (drafts) are being listed alongside completed reeplies – they shouldn’t be.
• When you add a reeply, it should be scored up by you automatically.

Fixed the first problem by adding the following to the relational info at the start of the article class:

has_many :complete_stories,
:class_name => “Story”,
:conditions => [ 'complete = ?', true ]

This is a top tip i just got from a forum, i’m going to look into doing more of this sort of stuff.

Now in the view, instead of saying

for story in @article.stories

I just say

for story in @article.complete_stories

Next – i want to call change_score from the create methods of story and article.  Here’s how I do it:

When a story is created, its points are set to 0.  There’s no StoryScore record for it at this stage.  When the user hits the ’submit’ button to submit the completed story, the submit method of the Story model is called.  I simply added a line to the submit method to call change_scores with 1, so it’s just as if the user hit the “score up” button when they submitted the story.

Next is to do the same for articles…ok.

I also want to check if there’s an article there already, and if so then score it up for the current user, instead of re-adding it.

Done that, in the add method of the article controller:

def add
#first of all, see if we already have an article with the passed url.
#if we do, we just want to score it up for the current user (and inform the user that we did it)
if (@existing_article = Article.find(:first, :conditions => ["url = ?", params[:article][:url]]))
@existing_article.change_score :user_id => session[:user].id, :points => 1
flash[:notice] = “This article is in Reeplies already.  It has been automatically scored up for you.”
redirect_to :action => ‘list’
else
@article = Article.new(params[:article])
@article.user_id = session[:user].id
if @article.save
flash[:notice] = ‘Article was successfully created.’
@article.change_score :user_id => session[:user].id, :points => 1
redirect_to :action => ‘list’
else
render :action => ‘new’
end
end#if else
end

One last thing – for each story, i want to list Reeplies in order of points.  This turns out to be super-easy!  I already set up a new associated called complete_stories, so i just add an ‘order’ parameter into that!  So my complete_stories association, in Article, looks like this:

  has_many :complete_stories,
:class_name => “Story”,
:conditions => [ 'complete = ?', true ],
:order => “points DESC, added_at DESC”

Sweet!  That’s a good point to end on today i think.

Converting movies to flv format.

I’ve decided to convert all  movies to flv (flash video) format before saving them, inna youtube stylee.

This page seems to spell out how to do that – there’s a lot involved.  Here goes.

Step 1:  Get the source for ffmpeg from subversion.  It involves a checkout, which seems odd…i don’t want to check anything back in (real open source coding is still scary to me), i just want a copy.  Ah well, i’ll follow the instructions.

I made a new folder outside of my rails app (i hope this was the right thing to do) and in it typed this at the command line:

svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg

This copied over a load of stuff.  Next step is to configure and compile it.  This sounds complicated.   First hurdle – the instructions are all for mac!

Mencoder looks promising as well…

OK, going with a different guide -

Step 1 – mencoder.  DLd and installed mplayer and mencoder and associated codecs.

Step 2 – test mencoder, with this, at the command line:

mencoder input.avi -o output.avi -oac lavc -ovc lavc -lavcopts vcodec=xvid:acodec=mp3 > output.txt

I get this error:

Cannot find codec ‘xvid’ in libavcodec…
Couldn’t open video filter ‘lavc’.
Failed to open the encoder.

After a bit of mucking around i got it to work (blob_woman2.avi is my test video, converted to output.avi using the lavc codec).  I had to remove the -oac lavc bit, and the -lavcopts vcodec etc bit as well.

mencoder blob_woman2.avi -ovc lavc -o output.avi  > output.txt 

It reduced to about 10% of previous size which is impressive, but i think it was an unoptimal avi in the first place.  Let’s try with a 237 meg divx movie now:  ok, it’s working….i’m getting messages about duplicate frames andthe occasional missing frame but it seems to be working away there…Finished!  237meg down to 130meg, that’s pretty good!  I had to specify an audio codec as well this time: lavc as well.

Following the instructions in the guide, the next thing to do is to incorporate it into rails.  Dmytro, who’s blog i’m using, sets up a helper method that takes a command line instruction, and generates a new type of exception (which is also defined but simply extends StandardError).  To be continued.

OK, got subversion installed, along with tortoise svn, which adds svn commands into windows explorer. I’m a little bit suprised that it (svn) doesn’t have a ‘proper’ interface – i was expecting something along the lines of visual source safe. I guess though that doing everything in windows explorer makes sense.

Anyway, the point of doing this was to get the movie playing stuff in. Let’s just try that command line before trying to work out how to do anything… it was

ruby script/plugin install svn://rubyforge.org/var/svn/flashplayrhelpr

Well, it did *something*. I got a load of stuff copied over, but then a cryptic message:

A C:\code\InstantRails\rails_apps\newspipe\vendor\plugins\flashplayrhelpr\install.rb
…
A C:\code\InstantRails\rails_apps\newspipe\vendor\plugins\flashplayrhelpr\MIT-LICENSE
A C:\code\InstantRails\rails_apps\newspipe\vendor\plugins\flashplayrhelpr\README
Exported revision 14.
Installing flash_player…
Plugin not found: ["svn://rubyforge.org/var/svn/flashplayrhelpr"]

What does that mean – do i have it or not? It was supposed (according to this guide) to make a folder called swf in public, but it didn’t. Goddammit, why isn’t anything ever easy? grrrr. ok, chill. The guide has a backup plan if the files don’t appear, which is to run this:

rake flash_player:install

which apparently makes the viewer available to use. Hmm. It didn’t complain when i ran this, but it didnt say it had done anything either. However when i did rake blahblah:install it complained that it couldn’t find blahblah so i guess it’s there in some form. Anyway, let’s try and use it.

I put this in my standard partial, which is the basis for all templates in my app:

<%= javascript_include_tag ‘ufo’ %>

then this in my movie/show view page:

<%= flv_player :file => “/data/movies/feynman.flv” %>

Here goes. Nope – i get this error message
undefined method `flv_player’ for #<#<Class:0×4815ad4>:0×4815aac>

which suggests it didn’t install.

Looking back at the messages, i think that the flash_player did install, but flashplayrhelper, which is a seperate plugin didn’t.

…some time passes…

Got it! After a helpful suggestion from the maker of flashplayrhelpr, Farooq Ali Farooq, I added the svn exe to my path variable, and forced reinstall – worked first time! Good old Feynman lecturing away there. Awesome.

There’s a little bit of tidying up required with the movie – it would be nice to be able to edit the associated details (title, summary etc) after posting for example. This isn’t essential though – what is essential is to be able to convert other video formats to flash video, so people can upload an avi, wmv etc. This is what YouTube does – you can upload various formats but everything is displayed in flv format.

This looks promising : http://www.danielfischer.com/2007/06/27/how-to-use-ffmpeg-to-convert-video-via-ruby-on-rails/

There’s quite a bit of pre-requisites but all stuff that would be useful i think: It seems perfect for my needs, dan even includes code for making a thumbnail. I’ll have a go with this (next post).

today – VIDEO

Can’t put it off any longer! It’s time to get that video functionality in there. As usual, its research time…

Preliminary research indicates that i should use a flash movie player, which can be embedded into html. This looks promising: http://flowplayer.org/ .

Toyed with the idea of rigging something up to show this in a page somewhere but thought i might as well do all the setup properly, since it’s going to be largely the same as for pictures except for a) where thumbnails come from (not so easy to generate for movies!) and b) the mechanics of actually SHOWing the movie, ie what happens on the movies/show view page.

So – that’s all set up, with blanks for generating and showing thumbnails, and for showing the movie.

Downloaded an flv (of the great Richard Feynman) from youtube using this handy website that let’s you rip flv movies off sites: http://keepvid.com/ . Also downloaded a free flv player just so i could check i actually had a working flv movie. I do.

OK, testing it – something weird’s happened – the movie has been saved as an ‘octet-stream’ – ah, wait, i think i was using imagemagick to save the full sized picture, i’ve just copied that over to movies, no wonder. I’ll change the controller method to just copy and rename a file rather than a picture. Actually i moved that to the model, let’s have a look: the current code says this:

#save the movie to newspipe/data/movies with the name ‘mov#{self.id}.extension’
new_filename = “mov#{self.id}.#{self.content_type.split(“/”).last}”
File.open(“#{RAILS_ROOT}/public/data/movies/#{new_filename}”, “wb”) { |f| f.write(self.file_data) }

Hmmm – that’s not using imagemagick! It’s just using File.open.  I think i need another way of saving a file without opening it.

In the meantime, been trying to get flowplayer working with a saved flv file.   Tested it outside of rails, it’s having an issue with security:

I’ve mucked about with the flash player settings, to no avail.

Ah wait, found it:  in the ‘global security settings’ panel, added the location “C:\code\useful stuff\flowplayer” (where i was testing it out) to the ‘allowed to access’ locations list.  So, i get to the movie player now but it doesn’t play properly – i just get a black screen in the player, and a stripy progress bar, whatever that means.

When i try it in newspipe, then i don’t get anything at all – not even the player.

There doesn’t seem to be anything about flowplayer in rails on the net.  Going to try a different line of enquiry for now.

OK, this looks promising:  another flash player, this time the JW FLV player.   Someone’s done a rails plugin for it here.  Here goes.  Ran this at the command line:

ruby script/plugin install svn://rubyforge.org/var/svn/flashplayrhelpr

nothing happened.  This looks like it might be a subversion (svn) – based plugin – maybe it’s time for me to install subversion finally.

A bit more on scoring

Playing around, just noticed that articles were disappearing after i clicked on them – this is left over from my old system where non-article links were awarded 0 points, and that was my system for deciding whether or not to show them. In hindsight this was a pretty stupid method – i should have marked them as link or article with a column in the articles table. As it happens i decided to add links to the article list anyway so it’s not necessary.

Anyway, i now need to make the _article_list show all articles, not just ones with a positive score. OK, done that, now i’ve noticed a new problem: the articles are paginated, 20 to a page. However, they’re not being sorted before pagination, only after pagination: so i have articles on page 2 with more points than the lower articles on page 1. This is probably because i’m doing my sorting in the list – i should probably have the sort method in the model somehow.

Put the <=> method in the model, still getting the same result though. Time to go and do some pagination research…

OK, that’s all working great now: here’s what i did. First, a controller method to generate a Paginator and the current set of pages: this replaces the scaffold-generated list method which did the same sort of thing but in a more rails-default way.

def list
#@article_pages, @articles = paginate :articles, :per_page => 20
#get the required page
page = (params[:page] ||= 1).to_i
items_per_page = 20
#get index of first item on current page
offset = (page – 1) * items_per_page
#count the items
item_count = Article.count

#create a paginator
@article_pages = Paginator.new(self, item_count, items_per_page, page)

#get the required subset of items
@articles = Article.find_page(:items_per_page => items_per_page, :offset => offset)
end

This calls an Article class method called find_page, which calls Article.find with various params:

def Article.find_page(params = {})
defaults = {:items_per_page => 20}
params.merge(defaults)
if params[:offset] == nil && params[:page_size] != nil
offset = params[:page_number] * params[:items_per_page]
else
offset = params[:offset] || 0
end
return Article.find(
:all,
:order => “points DESC, added_at DESC”,
:limit => params[:items_per_page],
:offset => offset
)
end

Note the :order param: this is telling the sql query to order by points (descending) and then by added_at (descending).

All that’s left now is to change the view code a little bit:

I made the previous/next page show up as text rather than a link if there’s no page to link to:

<% if @article_pages.current.previous %>
<%= link_to ‘Previous page’, { :page => @article_pages.current.previous } %>
<% else %>
<span>Previous page</span>
<%end%>
<% if @article_pages.current.next %>
<%= link_to ‘Next page’, { :page => @article_pages.current.next } %>
<% else %>
<span>Next page</span>
<%end %>

The rest is the same: for article in @articles etc.

One bit which i partly got from the web page that explained all this, replaces the previous page/next page bit with a nice
” << 1 2 3 4 >> ” set of page links:

<%= (@article_pages.current.previous)? (link_to ‘<<’, { :page => @article_pages.current.previous }) : “<<” %>

<% for page in @article_pages -%>
<%= link_to_unless((params[:page].to_i == page.number), page.number, :page => page) %>
<% end -%>

<%= (@article_pages.current.next)? (link_to ‘>>’, { :page => @article_pages.current.next }) : “>>” %>

I’ve put this into a partial called _page_links, which i call at the top and bottom of the list.

There’s a few enhancements i’d like to make – for example, to ‘list only my articles’, and when you go back from an article to go to the list page with that article. But i’m going to leave that for now and press on with finishing off the points: all that needs to be done now is to replace the ’score up/down’ text with arrow icons, which are greyed out if you’ve already done them on that article.

First, let’s make the link not show up if you’ve already marked it that way: (more conditional operator usage)

<%= (article.score_from_user(:user_id => session[:user]) != 1) ? (link_to “score up”, :controller=>”article”, :action => “change_score”, :article_id => article, :user_id => session[:user], :points => 1) : “scored up” %>

And similarly for ’scored down’. Next step is to simply replace the output text with image tags – i need 4 : two directions x two colours (green for up, red for down, grey for neither). I did these before, i think…

OK, done! The rather long conditional statements now look like this – i’ve edited them a bit to make the structure clearer:

<%= (article.score_from_user(:user_id => session[:user]) != 1)
?
(link_to image_tag(“/images/up_arrow_grey.jpg”, :alt => “up”, :border => 0), :controller=>”article”, :action => “change_score”, :article_id => article, :user_id => session[:user], :points => 1)
:
(image_tag(“/images/up_arrow_green.jpg”, :alt => “up”, :border => 0)) %>

Done a bit of tidying up of the graphics. What i’ve also been trying to do is to work out how to go to a given page: when you mark an article up or down, it sometimes goes onto a different page. I’d like in this case to switch the view to that page as well.

So, at the end of change_score i can write

redirect_to :controller => “article”, :action => “list”, :page => page

But the problem is getting a value for ‘page’. One thing i’m trying is to first get the ordered array of all articles, with

articles = Article.find(
:all,
:order => “points DESC, added_at DESC”
)

If i could then get the position of the desired article in this  list, i could work out what page it’s on from the ARTICLES_PER_PAGE constant i put in environment.rb:

page = (position/ARTICLES_PER_PAGE.to_i) + 1

But i can’t work out how to get the position.  I can do another sql search but that seems really wasteful:

article = Article.find(params[:id])
position = articles.index(article)
#this is returning one page too low!  why???
return (position/ARTICLES_PER_PAGE) + 1

This is inefficient and also consistently returns one page too low.  I don’t know why.   However, when i call it properly (instead of with a test id) from the end of change_score, it seems to work.  I’ve asked on the forums about the best way to get the page/position of an item, hopefully someone will come back with a more efficient method.

So – the points stuff is pretty good now.  That’s enough for today.

A join table for users and article scoring

So -

• A user can score many articles
• An article can be scored by many users, but only once by each user

So, it’s a habtm relationship, but not between users and articles, since user already “has_many articles” and articles already “belong_to user”. I’d better rename both – ‘articles’ will be ’scored_articles’ and ‘users’ will be ’scorers’. So the table will be called ’scored_articles_scorers’ What we want is a table that has scored_article_id (which uses has article_ids), scorer_id (which is users) and score (-1 or +1).

Here’s the migration:

class CreateScoredArticlesScorers < ActiveRecord::Migration
def self.up
create_table :scored_articles_scorers do |t|
t.column :scored_article_id, :integer t.column :scorer_id, :integer
t.column :score, :integer
end
end

def self.down
drop_table :scored_articles_scorers
end
end

Something odd – this is the same (with different names) as my articles_links migration, but the articles_scorers table has a unique id field as well, while articles_links doesn’t. Weird. Will need to keep an eye out for any problems associated with that.

Now, the tricky part (for me) – setting up the models. Should be similar to what we did for articles_links. That had this, in Article:

has_and_belongs_to_many :links,
:class_name => “Article”,
:foreign_key => “article_id”,
:association_foreign_key => “link_id”,
:join_table => “articles_links”

So, now, in Article we have

has_and_belongs_to_many :scorers,
:class_name => “User”,
:foreign_key => “scored_article_id”,
:association_foreign_key => “scorer_id”,
:join_table => “scored_articles_scorers”

And in User we have

has_and_belongs_to_many :scored_articles,
:class_name => “Article”,
:foreign_key => “scorer_id”,
:association_foreign_key => “scored_article_id”,
:join_table => “scored_articles_scorers”

So, let’s test it out! Restarted the web server. First lets do a ’score up’ link in the view, next to articles in the article list: “article”, :action => “score_up”, :article_id => article, :user_id => session[:user] %>

Next of course we need to do the controller method. Here’s my first stab – it’s incomplete for now – it doesn’t try to write the score into the scored_articles_scorers table, just to create a record and change the score:

def score_up
@article = Article.find(params[:article_id])
@scorer = User.find(params[:user_id])
@article.increase_score
@article.scorers < ‘list’
end

You’ll notice that the instance method being called “increase_score” is named differently to this controller method, “score up”. That’s because “increase_points” is a general kind of event that could happen for a variety of reasons, not just someone hitting the ’score_up’ button. However, ’score_up’ is a particular event where a particular user is involved. So, “points” is a dumb number, whereas a ’score’ is a more meaningful event with more information attached to it: who scored it for example.

OK – this works, it increases the article’s points and it makes a new record in the scored_articles_scorers with the proper article and user ids. What i need to do next is make it a) Check that the user hasn’t already marked it up b) Put the score ( 1 or -1) in the record.

I’ll do b first: this looks complicated. Reading up on it, it used to be simple: we’d use the method push_with_attributes (“push” is an alias for “<<”) and send the value along. push_with_attributes has been deprecated however – it looks like the proper way is to make an actual class that is instanced as a join table row. It says “we’ll discuss that in the next section- let’s have a look. ahhhh – these are the through tables that i keep seeing references to in the forums!

So, i should make a new class called Score, and rename scored_articles_scorers as ’scores’ (it’s a nicer name anyway). Then i say, in Article

has_many :scores
has_many :scorers, :through => :scores

and in User

has_many :scores
has_many :scored, :through => :scores

I still need to tell it somehow that ’scorers’ means “users” for example. Wait, we can do that with :source. So -

Article
has_many :scores
has_many :scorers, :through => :scores, :source => :user

User
has_many :scores
has_many :scored, :through => :scores, :source => :article

So, i need to set up the new model, change the database again and then change the above models.

OK, done that, and it’s working. Excellent. However i still need to add the score (1) to the row in scores.

(skipped a bit of reworking and re-re-working here) Here’s the final code:

In the view, i just call the change_score controller method with either 1 or -1:
<%= link_to “score up”, :controller=>”article”, :action => “change_score”, :article_id => article, :user_id => session[:user], :points => 1 %>

In the controller, i pass these params through to the instance method of the same name (most of the below is messages): def change_score
begin
@article = Article.find(params[:article_id])
rescue ActiveRecord::RecordNotFound
flash[:message] = “Couldn’t find article##{params[:article_id]}, redirecting to article list.”
redirect_to :controller => “article”, :action => “list”
else
flash[:message] = @article.change_score(params)
redirect_to :controller => ‘article’, :action => ‘list’
end
end

Then, in the model i do the logic: 

  def change_score(params = {})
new_points = params[:points].to_i
@score = Score.find(:first, :conditions => ["article_id = ? and user_id = ? ", params[:article_id], params[:user_id]])
if @score == nil
#@score not found
#no score exists from this user so make a new one then add the points
return_string = “Didn’t find score, making a new one”
@score = Score.new(:article_id => params[:article_id], :user_id => params[:user_id], :points => new_points)
self.points += new_points
@score.save
self.save
else #found something for @score
if @score != nil #@score was already there
#look for a nil and turn it into a 0 if present
@score.points = 0 if @score.points == nil
if @score.points != new_points
#score already there but points are different
#subtract old points then add new points
self.points -= @score.points
self.points += new_points
return_string = “changed your score for this article from #{@score.points} to #{new_points}”
#then change @score to have new points
@score.points = new_points
@score.save
self.save
else
return_string = “you already gave this article #{@score.points} point”
end #if
end #if
end #if
return return_string
end #method

Cool.  That’s enough for today.

More consolidation – bug fixing and improvements

These bullet points were left over from yesterday, and i’m adding a few more as we go:

• change what happens if someone adds an article that is already an article or a link: in both cases the story should have 1 point added to its score and the user notified either that ‘a link has been converted into an article’ or ‘the article is already present, you have added to its score’.
• put in some handling code for going back to ’show’ pages for things that have been deleted – “if @picture = nil” etc
  • This requires some exception handling – about time that i did this in rails!(see below)
    
  • Done for show picture – need also for story/show and article/show
    
  • done
    
• make pictures act as a link to the show picture page
  • done: code involves wrapping the image_tag as the first parameter to a link_to call:
  • link_to image_tag(“/data/pictures/thumbs/thumbpic#{picture.id}.#{picture.content_type.split(“/”).last}”, :alt => “Missing thumbnail”), :controller => “picture”, :action => “show”, :id => picture.id, :came_from => params[:action]
    
• If someone deletes a picture, we need to delete the file as well.
  • I’m going to put a delete method in the model for this, which will take an id and delete the full size and thumbnail pictures with that id. … didn’t work – ah wait, i forgot about the file extension. Now i come to think of it, this should be a class method (ie static) rather than an instance method as we might also want to use it to get rid of left over files that don’t have an object associated with them. Discovered that the way we do class methods in rails(ruby) is to put def Classname.methodname:

#expects options[:id] and options[:extension]
def Picture.delete_from_disc(options = {})
extension = options[:extension]
id = options[:id]
picpath = “#{RAILS_ROOT}/public/data/pictures”
if(File.exists?(“#{picpath}/pic#{id}.#{extension}”))
File.delete(“#{picpath}/pic#{id}.#{extension}”)
end
#now the thumbnail
if(File.exists?(“#{picpath}/thumbs/thumbpic#{id}.#{extension}”))
File.delete(“#{picpath}/thumbs/thumbpic#{id}.#{extension}”)
end
end

I then call that from the picture controller’s delete method, before calling destroy on the db object:

#delete from disc
Picture.delete_from_disc :id => params[:id], :extension => @picture.content_type.split(“/”).last
#delete from db
@picture.destroy

I think that’s the first time i’ve done a rails method that takes any parameters! woot.

One of the points above relates to exception handling: this is nice and easy: here’s how i’m dealing with the case when a requested picture (in show picture) doesn’t exist: found out that you can’t ask for ‘action_name’ in a view, you need to ask for params[:action]. (that reminds me, going to put params output into a debug partial and add it to the _standard layout. OK, done that.)

def show
begin
@picture = Picture.find(params[:id])
rescue ActiveRecord::RecordNotFound
flash[:message] = “Couldn’t find picture##{params[:id]}, redirecting to article list.”
redirect_to :controller => “article”, :action => “list”
end
end

Nice and simple – if we can’t find the picture we go back to the article list. Ideally we could go back to the story the picture belongs to, but that requires knowing the story, which we don’t ordinarily pass through – normally we get the id of the story from the picture’s story_id, but of course we don’t have a picture so we can’t.

That bit of exception handling has got me onto a bit of user testing – found that if you log out while in the ‘edit_story’ page, it crashes looking for an id of nil. Fixed that by putting a bit of checking code in edit_story, which redirects to show_story if the @story.user is different to the session[:user].

Now i have a new, similar problem – going to show picture after doing this, if you hit back it says “no action responded to edit”. Hmm, it should be going back to show, not edit. Let’s have a look. Ah yeah – when i go back from picture, (to story), i wasn’t specifying the controller (which should be ’story’ of course). Fixed now, along with a bit of other weirdness to do with swapping between show_picture and edit/show story.

Now, where was I? Oh yeah, putting in the exception handling for show_story and show_article. Both will just take you back to the article list if we can’t find the item. OK, that’s done. Looking at ta-da lists, a successful rails app, they have a standard ‘page not found’ page, which has a nice error message and the option to go back. Will look at doing that at some point.

Made a slight change to the show/edit story page – they now show the article that the story is a reply to – this should be useful reference for anyone writing/viewing a story.

•  Allow user to delete stories – this should delete any associated pictures and movies.

Got this in with a little help from a friend.  It works in quite a cool way:

First we add a :dependent parameter to the has_many: pictures line in the story model.  This says “when a story is destroyed, do ’something’ to it’s children (the pictures).  In this case the thing we want to do is :destroy  the pictures (the most common use of :dependent).  So, :dependent => :destroy.  “:destroy” is a fixed option symbol, it’s not like a method call into the picture model or controller or anything like that.  It’s like saying “for picture in @story.pictures do picture.destroy”.

However, that’s not enough:  calling destroy on a picture only gets rid of the db record, we need to get rid of the files as well.  We’ve got a method to do that, so in the picture model, we can say ‘before destroying, call this instance method”.  We do that by writing, in picture.rb:

before_destroy :delete_from_disc 

Simple!  Btw, the instance method in picture looks like this:

  def delete_from_disc
picpath = “#{RAILS_ROOT}/public/data/pictures/pic#{self.id}.#{self.content_type.split(“/”).last}”
thumbpath = “#{RAILS_ROOT}/public/data/pictures/thumbs/thumbpic#{self.id}.#{self.content_type.split(“/”).last}”
if File.exists?(picpath)
File.delete(picpath)
end
#now the thumbnail
if File.exists?(thumbpath)
File.delete(thumbpath)
end
end

Note that we don’t call self.destroy – that’s done elsewhere, after this method completes.

Another thing i learned last night was the ’skinny controller, fat model’ principle.  That states that the controller shouldn’t ever be more than a traffic cop – it shouldn’t do stuff to objects (at least as little as possible anyway), it should just route the program around between other controller, actions and views.  That means that at the moment, where i have in a controller action, some setup code like

story.added_at = DateTime.now.to_s
@story.user_id = session[:user].id
@story.complete = 0

I should really be doing this in the model.  So, I moved that into an initialize method in Story.  Discovered that if you do this you need the line super before doing anything, to call the ActiveRecord initialize method, which does the railsy setup.  However, my next problem is this line:

self.user_id = session[:user].id

In the model, we can’t seem to access the ’session’ hash – i’m getting an ‘undefined variable’ exception.  After a bit of research it seems like the model deliberately can’t access the session.  So, i guess i’ll need to pass the user through to the constructor.   So, this is the new story constructor:

def initialize(params = {})
super
self.added_at = DateTime.now.to_s
self.user_id = params[:user]
self.complete = 0
end

OK – time to move some more stuff into the story model, like what happens when the story is submitted.  Discovered that even when you do your changes in the model instead of the controller, you still need to call “save” to get them into the database.  New submit method, called by story.create:

  def submit
self.complete = 1
self.points =  1
self.added_at = DateTime.now.to_s
return self.save
end

Again, nothing new, just getting the model to do the work.  Let’s do the same for article controller.

Having problems with this:  here’s my new controller method:

def create
@article = Article.new(params[:article], :user => session[:user])
if @article.save
flash[:notice] = ‘Article was successfully created.’
redirect_to :action => ‘list’
else
render :action => ‘new’
end
end

and the new constructor:

 def initialize(params =(), *options)
super
if self.title == “” && self.url != “”
self.title = (Hpricot(open(@article.url))/”title”).first.inner_html
end
self.added_at = DateTime.now.to_s
self.user_id = :user
self.points = 1
end

I’m having problems getting everything through properly to the constructor.  I think for now i’m going to go back to doing the user_id stuff in the controller and fixing it later when i understand what’s going wrong.

OK – that’s enough controller slimming for now.

Next task:  Check validation code is doing all it should – a picture should have a filename, for example.

Done: Picture validates_presence_of :title, :filename.  Think that’s it for now.

Just one bullet point left to do:

• change what happens if someone adds an article that is already an article or a link: in both cases the story should have 1 point added to its score and the user notified either that ‘a link has been converted into an article’ or ‘the article is already present, you have added to its score’.

I’ve been thinking about this and i think that any link added to a story should be a fully fledged article as well – there’s no reason not to really.  In order to implement this, all i need to do is to set a link’s score to 1 instead of 0.  Sweet.  Actually, it’s even easier, i just use the article constructor that i made earlier which sets the score to 1 anyway.  Even better.

OK, so next i check whether the article is there already, and if it is then i add a point to it rather than make a new one.

That’s done.  But, i noticed that i can add a link to an article even if it already has it as a link.  So, i need to do a further test after seeing if the article exists already:

      if @original_article.links.include? @article
flash[:message] = “Link already added to this article!”
else
@original_article.links << @article
flash[:message] = “Added article #{@article.id} to article #{@original_article.id}’s links.”
end

This works, but i think this is ‘fat controller’  – i should move this to the model really.  I could probably do something clever with callbacks (the model equivalent of filters).  Hmm.

Anyway, cleanliness aside that’s the bullet points all sorted!  Cool.

This dealing with points has reminded me that i need to get the points system up and running properly so that people can add points to articles.   I need to think about the schema for this a bit:

• Each user only score a particular article either up or down, once.
• They can change their mind, and swap n up for a down or vice versa.
• A user can score many articles.
• An article can be scored by many users.

So what we’re talking about is a join table between user_id and article_id, which has an extra field that holds the score somehow – either “up” or “down” or -1 or 1, or something.  Hmm, i’ll think about this over the weekend.

A bit of consolidation required

Getting pictures working turned into a much harder process than i thought. Before i put movies in, i think i need to do a little bit of tidying and general improvement. Such as (still to do are in red)

• add textilize functionality to the relevant displays
  • done
  • Discovered that textilize(blah) automatically puts the output in a <p> element which i didn’t want. Fortunately there’s a textilize_without_paragraph() method as well. Sweet.
• add some half decent styles and make the various view components use them appropriately
  • list page- looks ok
  • show page – article list is nasty.
  • Story and link titles should use the same style, now called “title_list
  • summaries and comment bodies should use the same style, “summary_list”
  • any “added by fredd 2 minutes ago” bit should now use “poster_name”
  • titles on pages now use”title_show”
  • summaries on pages now use “summary_show”
  • added classes “tags_show” and “body_show” for any tags and bodies
  • put options links (such as “back”, or “go to list”) in style ‘option_link’
  • made sure that list headers like Comments etc use style ‘list_header’
  • Generally put everything in an appropriate style. The styles themselves could still do with a bit of work.
    
• make an edit story page that isn’t fugly.
  • It looks a bit nicer. Made it less table-y and more list-y. Made show_story page nicer as well, including only showing tags, summary, body, pictures or movies if there are any of them actually there.
• add some more links – eg if you are the user who added a story you can move from show_story to edit_story
• also some more ‘back’ links (and fix them- eg’back’ on the add_link form takes you to the list.)
  • can use existing save_page partial and session[:return_to] for this.
  • Actually, i shouldn’t as the log in and out uses that already so there’ll be crossover confusion between logging in and out and moving around.
• do some controller refactoring, for the sake of the urls (and general cohesion) – there shouldn’t be a NewspipeController for example, otherwise the url will be www.newspipe.com/newspipe which looks a bit wrong. Should split the current controller actions up into ArticleController, StoryController, LinkController, CommentController, PictureController. Maybe eclipse can help with this?
  • Eclipse can help a bit with a global search and replace – this is very useful for dealing with partials.  Doing that at the moment – putting partials in the appropriate view classes.
  • OK – that took ages.  Here goes.
    • The stylesheets aren’t being picked up.
      • Ah – it’s not using the layouts, that’s why.
    • Spotted a couple of error, to do with calling methods (probably calling the wrong controller)…ah, i see what it is: tricky!  The ‘list_story’ partial, even though it lives in the story view folder, is calling the show method for ‘article’, because the partial is inside the article view when it is listed.  I’m going to go through all the partials and make them specify a controller.
    • Couple of other things.  Think it’s all sorted out now.
    • Also noticed that the title code was broken – did a big set of nested case statements which analyse controller_name and action_name.
• change what happens if someone adds an article that is already an article or a link: in both cases the story should have 1 point added to its score and the user notified either that ‘a link has been converted into an article’ or ‘the article is already present, you have added to its score’.
• put in some handling code for going back to ’show’ pages for things that have been deleted – “if @picture = nil” etc
  • This requires some exception handling – about time that i did this in rails!
• make pictures act as a link to the show picture page
• If someone deletes a picture, we need to delete the file as well.

Aside – a note on automatically getting the method that called the current method (useful for going back):

You can use caller inside the method, or perhaps params[:action].I wouldn't do it that way though. Just allow the method to take aparameterspecifiying if that extra bit of code should be run.

Monday – still having rmagick problems.

Got rmagick installed – the problem was the the latest windows version of the gem was actually broken and nobody bothered to mention it or change it. Only found out off a forum that it was broken. They pointed me at an earlier one which did install fine in the usual way.

To get it to work, i added these lines to environment.cfg:

require ‘rubygems’
require ‘RMagick’
include Magick

Next, according to instructions here, i added this line to my newspipe controller:

imagemagick_for ‘/data/pictures/’, :cache => ‘/data/pictures/cache’

ARGH WORDPRESS JUST FORCED ME TO SIGN IN AND LOST LOADS OF MY WRITING!  THERE WAS MORE STUFF HERE!

god damn it.  I had loads of stuff about image displaying hassles here.  OK, here’s a summary instead.

I have the above in the controller and this in the view:

<%= imagemagick_tag “pic#{picture.id}.#{picture.content_type.split(“/”).last}” %>

It’s making the picture name (eg pic32.jpeg) ok but isn’t displaying it because it keeps looking in appname/imagemagick for the file.  It’s like it’s ignoring the imagemagick_for line.  Maybe it’s in the wrong controller?  Maybe it needs to be in a special controller for pictures or something?

image_tag is working now, since i found out that any data that i read and write in my app needs to be in the appname/public folder.  What i’m doing now is using imagemagick to save a thumbnail of the picture alongside the picture file, prefixed with the word ‘thumb’: this is a method of Picture so lives in picture.rb:

def write_to_disc
#save the picture to newspipe/data/pictures with the name ‘#{self.id}.extension’
new_filename = “pic#{self.id}.#{self.content_type.split(“/”).last}”
File.open(“#{RAILS_ROOT}/public/data/pictures/#{new_filename}”, “wb”) { |f| f.write(self.file_data) }
#now save a thumbnail version as well
thumb = MiniMagick::Image.from_file(“#{RAILS_ROOT}/public/data/pictures/#{new_filename}”)
thumb.resize “200×200″
thumb.write(“#{RAILS_ROOT}/public/data/pictures/thumbs/thumb#{new_filename}”)
return true
end

I’m calling this as part of the create_picture method in the controller.

When it comes to displaying the pictures, i’m showing the thumbs in the view with:

<%= image_tag “/data/pictures/thumbs/thumbpic#{picture.id}.#{picture.content_type.split(“/”).last}”, :alt => “Missing thumbnail” %> 

So, i’m showing the name and thumbnail of every picture belonging to the story.  Currently in a very messy way.  I need to a) sort the messiness out and b) make the name and the thumbnail link to that picture’s ’show_picture’ page.  Shouldn’t be hard.  c) add a ‘delete picture’ link to the side of the picture name.

a) is done.

b) Remembered that i was going to show pictures and movies in the viewer to the right of the pic and movie list.  Hmm.  Thinking about that more, it places a severe limit on the size of the picture.  I think it’s best left as a movie viewer, as limiting the size (on the screen) of movies is reasonable.  So, pictures get their own page.

So, the controller’s pretty simple: we just need to get something into @picture:

  def show_picture
@picture = Picture.find(params[:id])
end

Next, the view.  Ideally there should be the same view_picture page whether you’ve come from show_story or edit_story.  That means that the ‘delete picture’ option should only be available if session[:user] is the same as @picture.user, and if we came from edit_story (since deleting pictures falls within the bounds of editing a story, i don’t want to give the option if the user was just viewing the story.  if i change my mind about this later it’s easy to alter the code).

It also means i need to remember where the user came from (show_story or edit_story) – this will need to go into params when show_picture is called.  The second bit was easy:  when i call show_picture i pass through a parameter called :came_from, which is set to the action i’m calling from.  (is this passed through automatically in rails?)

Next, the actual deletion.  Looking at the app, i realised i don’t have any deletion capabilities anywhere at the moment – articles, stories etc can’t be deleted.  Need to go back and look in another app to remember how to do it…

Ok, it’s easy – we just have an action called destroy which calls destroy on the object in question and redirects to where we want to go.  I’m going to do a different destroy action for pictures, movies, articles etc as the logic of what to do next may differ.

OK, done that, nice and easy (damn it’s good to be making progress again).  The controller action (note that we need to make a note of what story to go back to before we destroy the picture, but i can just pull the place to go back to out of the params – this means that i can add the ability to delete pictures from other places without this breaking):

def delete_picture
@picture = Picture.find(params[:id]).destroy
@story = @picture.story
@picture.destroy
flash[:message] = “Picture #{params[:id]} deleted”
redirect_to :action => params[:came_from], :id => @story
end

and the link in view

<%= link_to ‘Delete this picture’, :action => ‘delete_picture’, :id => @picture, :came_from => params[:came_from] %>  

Added that next to the picture title in the edit_story page as well. (substituting picture for @picture).

Cool.  So i can now add pictures to a story, show a list of thumbnails, view them full size, and delete them.  It’s 8.30 so that seems like a good time to stop and have dinner.

Adding pictures (continued)

OK – so yesterday i got photos working in the other photo_holder app, and more importantly got my head (mostly) around what it’s actually doing. First thing today is to get that into newspipe.

There’s a slight model change required to picture – we need to change the ‘blob’ field (for binary_data) to longblob – looking at photo_holder, it can only store the first few hundred k of any image, which isn’t anywhere near enough. Changing it to longblob in the db fixed it, so i just need to find out the rails word for ‘longblob’.

Found this useful bit of info:

In MySQL 4.1 and above, you can specify the length on a blob or text by using size. It’ll automatically expand to the right type. Example:

create_table :files do |t|
  t.column :data, :binary, :null => false, :size => 400000
end

So, i’ll do a migration to drop the binary_data column and replace it with another (which i’ll just call data actually) which is bigger. I’ll give it a limit of 5 meg, that should be enough for any typically sized jpg. While i’m at it, i’ll do the same for movies but give a limit of 500 meg. Need to remember to put these limits into validations in the model.

Hmm…the migration seemed to run ok but doesn’t seem to have changed the db. Ah wait – i’ve still got my development database set to that test one i set up yesterday to check my migrations were all in order.

…

Just had some problems in eclipse – found out that when i did the import of newspipe yesterday i didn’t set it up to run in the same folder properly (which was why my breakpoints weren’t working – i was putting them in the originals and the copies were running). Deleted the project, re-created it and this time set it up to work out of the original folder. So now i can debug with breakpoints, and i’m happily working in eclipse now, which is a bit nicer and definitely more professional.

That also means i’ve got no excuse for not doing unit tests :) Will cover that later though.

For now – i’ve got the new version of the ‘new_picture’ form in and working. Next i need to add that method to the picture model and then change the create_picture action.

Done those – seems to have worked. Next i just need to add a render_picture method which renders the picture to the browser. Then, i can add a line to ‘edit_story’ and ’show_story’ to output the pics. Here goes.

Worked first time! Amazing.

Having said that, eclipse just crashed trying to render a large picture and froze my browsers out until i killed the process. That’s not very good. I’ll try again.

Hasn’t crashed this time but it’s R E A L L L Y S L O O O oh wait, it has crashed again. When it crashed, it seemed like my HD went into overdrive, and according to task manager my cpu usage was also at 100%. How does that happen? Solid IO and solid CPU? Seems crazy. Let’s try running it the oldschool way, with a command line mongrel_rails server. uh oh – mongrel’s complaining. I bet the server eclipse started is still running.

can’t see anything obvious in task manager’s process bar. Killed a few things anyway. No joy. Tried mongrel_rails stop but it doesn’t seem to think it’s running. Argh.

Going to reboot. What a pain in the ass. Ah well, get a chance to read some more of “Surely you’re joking, Mr Feynman”.

OK, back again.  Running oldschool now, seems fine.  EXCEPT that there’s a problem with the pictures – i’m still only getting some of the picture – at a rough guess, about 80-100k of it.   The photo_holder app had this problem until i went into the DB and changed the type from ‘blob’ to ‘longblob’.  Putting the size in the migrate should have made a field to hold big enough blobs but it clearly hasn’t worked.

Actually, thinking about this some more, i probably shouldn’t put pictures in the database at all – i should put their id and details in the database, then save the picture in an app folder with the name renamed to id_number.jpg or whatever.  That would be smarter.  I can add save and load methods to the picture model.  This will definitely be better for movies as well, where i really don’t want to be pulling 50 meg files out of a database.   Putting a thumbnail in the db should be fine though.

So, the new plan for pictures is to

• take the uploaded image
• rename it with @picture.id and save it somewhere
• reduce it to a thumbnail (keeping the same name)
• put the thumbnail into the db

Then, it will be the thumbnail that’s displayed on the edit_story and show_story pages, but the thumbnail and title act as a link to show the full-sized picture.  Or, maybe i just show the full sized picture on those pages.  Either way, saving a thumbnail seems like a good idea.  Initially, let’s not worry about thumbnails and try to just save and retrieve the picture – i can go back to displaying just the title on the edit page.

So – i need to do a migrate to drop the data fields altogether.  Then, edit the model so that it doesn’t save into the data field., but holds onto the data instead.  I just did this by making it write into an attribute of the object that doesn’t have a corresponding db field :”file_data”.  Then, i call a write_to_disc method from the controller, which i’ll need to add now to the model.

OK – this is the Picture model now:

class Picture < ActiveRecord::Base
belongs_to :story
belongs_to :user

validates_presence_of :title

attr_accessor :file_data

def picture_file=(input_data)
#unless params[:photo].blank?
# @photo.update_attributes(params[:photo])
#end
self.filename = input_data.original_filename
self.content_type = input_data.content_type.chomp
self.file_data = input_data.read
end

def write_to_disc
#save the picture to newspipe/data/pictures with the name ‘#{self.id}.extension’
new_filename = “#{self.id}.#{self.content_type.split(“/”).last}”
if self.file_data.write(“C:/code/InstantRails/rails_apps/newspipe/data/pictures/#{new_filename}”)
return true
else
return false
end
end

end

This isn’t working – when i call write, it crashes and complains because i’m calling ‘write’ on a string.  But i’m setting file_data equal to the return from read, which i thought was data.  It makes sense to get something with ‘read’ and then call ‘write’ on it…but it’s not working.

As an experiment i tried setting file_data equal to the whole input_data object, but i think that’s just a hash.  It doesn’t save anything anyway, although it is called fine.

Had a recommendation off a forum to set file_data equal to a TempFile object, using input_data.  Trying this with

self.file_data = ::TempFile.new(input_data)

but get back an error saying “

uninitialized constant TempFile

I’ve got

require “tempfile”

at the top of the model file…

Tried something else and have cracked it!  YAY.  I’m using File.open, which if you give it the params “wb” means “open for writing as a binary”:

File.open(path, “wb”) { |f| f.write(self.file_data) }

Very pleased.  Currently ‘path’ is made up of a hard coded rails location and local path, plus the autogenerated file name.

Just improved this by using RAILS_ROOT which returns the location of the newspipe folder (in a really roundabout way, by going in and out of config a few times!)

So, my model methods are now as follows:

 def picture_file=(input_data)
self.filename = input_data.original_filename
self.content_type = input_data.content_type.chomp
self.file_data = input_data.read
end

def write_to_disc
#save the picture to newspipe/data/pictures with the name ‘#{self.id}.extension’
new_filename = “pic#{self.id}.#{self.content_type.split(“/”).last}”
File.open(“#{RAILS_ROOT}/data/pictures/#{new_filename}”, “wb”) { |f| f.write(self.file_data) }
return true
end

Next, i want to

• retrieve the picture from the data folder, and show it on the show_picture page.
• make a thumbnail of the picture and put the thumbnail into the database.  Or maybe it would be best to just save it on the hd as well and keep the db access fast and minimal.

Trying to install rmagick, which allows lots of easy image manipulation (such as displaying thumbnails) and it’s just been a constant struggle.  Got it installed but mongrel_rails complains about a missing dll when it tries to load.  gah.  Stopping now and going to the pub.